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Executive Summary 


This initiative was launched in September 2014 by the University of Chicago Impact (UChicago Impact) in 
partnership with Chicago Public Schools (CPS), the Lefkofsky Family Foundation, and the Academy for Urban 
School Leadership (AUSL). 


The Success Project utilized the 6tol6 curriculum for sixth-, seventh-, and eighth-grade students. The 6tol6 
curriculum, described by program staff as the cornerstone of the program, is a college-readiness and social capital- 
based curriculum to provide low-income students with the knowledge, beliefs, and skills to select, enter, and 
successfully graduate from college. An additional component of the Success Project in the 10 Success schools was 
a dedicated Success coordinator in each school. The coordinators were full-time staff at the schools. They were 
tasked with three core responsibilities: 1) teaching the 6to16 curriculum, 2) monitoring and analyzing students’ 
attendance, grades, behavior, and middle grades on-track status (a metric used in grades 3-8 to indicate if students 
have an attendance rate of 95 percent or higher and reading and math grades of C or better),! and 3) assisting with 
the high school application and selection process. Coordinators collaborated with administrators, teachers, and 


other staff members to support students in all these areas. 


The Success Project was fully implemented in 10 neighborhood elementary (K-8) schools in CPS, henceforth called 
the “Success schools.” Some elements of the Success Project were also implemented in AUSL schools serving 
middle grades. (AUSL adapted the program to their model, largely incorporating the 6to16 curriculum and dividing 
program management across staff, rather than with an appointed Success coordinator.) This report is the result of 


a formative evaluation of the 10 Success schools, undertaken by the University of Chicago Consortium on School 


1 Throughout this document there are references to middle grades on-track. This is a metric used in CPS for students in grades 3-8. We 
use the same definition as CPS, but we call it “middle grades on-track” because we only apply it to students in grades 6-8. Students are 
on-track in grades 3-8 when their attendance is 95 percent and their reading and math grades are C or better. 
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Research (UChicago Consortium), to provide data to inform programmatic improvements over the course of the 
three-year implementation. To understand how the program was implemented in the 10 Success schools and to 
understand changes within the theory of action over time, we conducted interviews with and observations of the 
Success coordinators. In the first two years, all coordinators were interviewed about their experiences. Additionally, 
we also interviewed all other program staff, including the former Executive Director of UChicago Impact who was 
responsible for launching the program in partnership with CPS, UChicago Impact’s Director of College Success and 
Assistant Director of College Success who were responsible for managing the program and supervising the 
coordinators, and other UChicago Impact staff and consultants who provided support and expertise to implement 
the program. At the end of 2014-15 (Year 1) and 2015-16 (Year 2), we provided a memo summarizing the data and 
findings from that program year to Success Project staff. This is the third and final report, and includes data from 
the program’s inception through its conclusion. This report also examines two sets of student outcomes: students’ 
achievement in the middle grades and the high school choices they made. These analyses do not provide causal 
evidence about the program. Rather, they document what occurred in the first two years. Insufficient time has 
passed to conduct an evaluation of program effects on some of the key student outcomes that the program was 
intended to influence. (Only one cohort has finished one year of high school and that cohort received only received 
a half of the a year of the program; thus, we cannot offer any insight into students’ performance after they left the 
Success schools.) Rather, the student outcome analyses are intended only to provide initial data following program 


implementation. 


Implementing the Success Project 
Success Project coordinators and staff launched the program with a vision of how the program would operate in 
schools. They soon learned that they needed to make modifications to their program, in light of students’ and 


schools’ needs as well as contextual factors that would influence the program’s success. 


6tol6 

Within their 6to16 work, the Success Project staff and coordinators learned that identifying and protecting time to 
teach the 6to16 curriculum within their school schedules was a critical step in implementation. The Success Project 
staff and coordinators learned that their schools had very different contexts and that having flexibility to find the 
right time to teach the curriculum, as well as the flexibility to either deliver the curriculum to a whole class or to a 
small group of students, was key to successful implementation in their schools. Principal buy-in also helped protect 
6tol6 time. In addition, the staff and coordinators learned that the curriculum needed modifications to be relevant 
and engaging for students, and that successfully managing student behavior impacted the degree to which 6tol6 


could be successfully taught. 


Grades and Attendance 

Success coordinators found that the district’s priorities and accountability framework—which prioritizes students’ 
test scores—influenced their schools’ priorities and how willing their schools’ administration and staff were to 
engage in efforts to improve students’ grades and attendance. Coordinators also found that moving more students 
to earn As and Bs was often a difficult and delicate task, because it revolved around teachers’ grading practices and 
students’ content knowledge. Last, while establishing buy-in and trust from administrators, teachers, and students 
was important for all of the coordinators’ work in their schools, the amount of buy-in and trust that coordinators 
cultivated greatly affected the degree to which coordinators were able to engage in school-wide, classroom, and 


individual interventions around grades and attendance. 
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High School Application and Selection 

Success coordinators and program staff learned that the workload required to get every student to identify, apply 
to, and gain admittance to “good match and fit” high schools—and then to select a “good match and fit” high 
school—required a tremendous lift. Buy-in and support from families and counselors was critical throughout the 
application and selection process, since counselors had to do part of the application work, including sending 
students’ transcripts and test scores, and families would make the ultimate enrollment decisions. Finally, Success 
Project staff and coordinators first approached school “fit” as a school’s focus, course offerings, and activities; they 
later expanded school “fit” to include a student’s commute and safety, as well as their families’ concerns, 


limitations, priorities, and history with the school. 


Initial Student Outcomes 

Most of the early student-level outcomes that the program targeted in the middle grades showed signs of 
improvement. Students’ attendance and middle grades on-track rates point to positive changes in the Success 
schools, especially in Year 2. Students’ reports of their own priorities for school work and how they perceived their 
peers’ behavior toward academic achievement improved compared to prior years. They also reported that they 
connected with teachers better and that teachers supported them in their academic achievement as well. Students 
also reported feeling better connected to the school community. These are some positive signs that the culture in 


these schools changed during the course of the project. 


Students’ GPAs did not show much improvement. As coordinators reported in both years, this was the task where 
they struggled the most. To accomplish all of their work, coordinators needed to develop and implement school- 
wide systems and structures and to gain trust and buy-in from administrators, teachers, and students. It took time 
for the Success Project to start building these systems and to cultivate trusting relationships and support for the 
program. Though there are initial indicators that student outcomes and school climate may have been improving, 


it would likely have taken more time for the program to expect to see meaningful improvements in students’ GPAs. 


Fewer eighth-grade graduates attended career academies and neighborhood high schools and more attended charter 
high schools after two years of Success Project implementation. One of the program’s goals was to find a “good 
match and fit” high school for each student to make a successful transition to high school. The high schools attended 
by Success schools’ students had similar ACT scores, but lower graduation rates than the high schools attended by 
prior cohorts of Success schools’ graduates. These changes were no different from what was observed for graduates 
of the comparison groups of schools we included in our analyses. While the information on the type of high schools 
students enrolled gave us an initial picture of the high school choices and how they have changed, it did not tell us 
much about the particular qualities of those choices or how students would perform while in high school. Only one 
cohort of eighth-grade graduates had attended high school for a full year at the time we ran our analyses, and that 
cohort only received a partial year of the intervention; the second cohort is currently in their ninth grade year — 
with this limited data we cannot study their outcomes in high school. Their performance will be a good measure of 


whether students found a “good match and fit” high school. Unfortunately, it is too early to study these outcomes. 


With only 10 schools and given the newness of the program, it is difficult to measure whether the changes in Success 
schools were larger than what other schools experienced without this kind of intervention. Most interventions need 
three to five years to fully see measurable effects on students’ outcomes. However, these changes point to 
movements in these schools in the direction the Success Project hoped to accomplish. The accomplishments and 
challenges from two years of Success Project implementation offer insight for other programs that seek to influence 
these outcomes: students’ “high school and college knowledge,” study habits, and social and cultural capital; 


students’ grades and attendance; and high school choice. 
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Introduction 


The Success Project was launched in September 2014 by UChicago Impact? in partnership with CPS, the Lefkofsky 
Family Foundation, and the AUSL. Underlying this initiative was a belief that many urban public middle school 
students in the United States enter high school without sufficient social supports and academic skills to succeed in 
high school and college. The program addressed this gap through a focus on three key areas: 1) Establishing strong 
grades and attendance; 2) building social and cultural capital; and 3) improving high school selection processes. 
These focus areas were addressed through a course taken by all middle grade students—called 6tol6—and the 


engagement of a Success coordinator based at each school. 


In Year 1 (2014-15), the Success Project was implemented in 10 neighborhood elementary (K-8) schools in CPS, 
henceforth called the Success schools. Integral to the Success Project’s approach was the placement of a full-time 
coordinator in each school, henceforth called Success coordinators or simply coordinators. These coordinators 
joined the Success Project with a wide array of backgrounds and expertise. Approximately half of the coordinators 
had teaching experience. Most of the other coordinators had been working for non-profit programs in schools, for 


example working as college access counselors or mentors, and one coordinator came from the private sector. 


The Success Project was also implemented in 23 AUSL schools in Year 1, but they did not have a full-time 
coordinator implementing the initiative in their schools. Instead, AUSL staff members received training and 
professional development to implement the Success Project in their schools. In Year 2 (2015-16), the Success Project 
was implemented in the same 10 Success schools and the same 23 AUSL schools; additionally, four more AUSL 


schools joined the initiative. 


2 The UChicago Consortium and UChicago Impact are separate units within the University of Chicago Urban Education Institute (UEI). 
For more information on UEI, including the UChicago Consortium and UChicago Impact, please visit https://uei.uchicago.edu/ 
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The project was intended to last for three full academic years and to have coordinators in the Success schools for 
three full years. As the program was launching, there were some challenges that prevented the coordinators from 
starting in schools right away. While the project did begin in August 2014, coordinators did not enter their schools 
until the beginning of the second semester (late January) in Year 1 (2014-15). The program came to an early 
conclusion during the spring of Year 3 (2016-17). As a result, this formative evaluation report focuses on the first 


two years of implementation in the Success schools. 


Program Model 

The Success Project utilized the 6tol16 curriculum for sixth-, seventh-, and eighth-grade students in all schools 
implementing the initiative. The 6to16 curriculum was described by the program staff as the cornerstone of the 
project and it was originally developed by the University of Chicago Charter School. 6tol16 is a college-readiness 
and social capital-based curriculum, designed to provide low-income students with the knowledge, beliefs, and skills 
to select, enter, and successfully graduate from college. As described by the initial developers of 6to16, the 


curriculum and modules were designed to provide students with opportunities to develop: 
e A belief in their ability to graduate from college and be committed to this vision; 


e A path, knowing the process for getting into and graduating from college and demonstrating that they are 


on that path; 
e A readiness (in skills and competencies) to succeed in high school, college, and beyond; and 


e A connection, exercising agency and ownership over their educational process and identifying and 


accessing social capital to support their goals. 


An additional component of the Success Project in the 10 Success schools was a dedicated Success coordinator in 
each school. The coordinators were full-time staff at the 10 schools. They were tasked with three core 
responsibilities: 1) teach the 6to16 curriculum, 2) monitor and analyze students’ attendance, grades, behavior, and 
middle grades on-track status, and 3) assist with the high school application and selection process.? Coordinators 


collaborated with administrators, teachers, and other staff members to support students in all three areas. 


Project Goals and Outcomes 

In the short-term, the goals of the program were 1) to improve grades and attendance to increase on-track rates in 
the middle grades, and 2) to increase the percent of eighth-graders who applied to selective enrollment and other 
“good match and fit” high school options. The program staff and Success coordinators described a “good match 
and fit” high school for a student as a school that matched a student’s academic qualifications (match) and gave a 
student the opportunities to explore and work toward his/her interests and career goals (fit). In the medium-term, 
the expectation was that increasing students’ middle grades on-track rates would lead to a better transition to high 
school, increased success in high school, and higher high school graduation rates. The desired long-term outcome 


of the project was to produce higher college enrollment and graduation rates for students in these Success schools. 


3 Throughout this document there are references to middle grades on-track. This is a metric used in CPS for students in grades 3-8 called 
‘Grades 3-8 On Track’. We use the same definition as CPS, but we call it “middle grades on-track” because we only apply it to students 
in grades 6-8. Students are on-track in grades 3-8 when their attendance is 95 percent or higher and their reading and math grades are 
C or better. 
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Another goal of this three-year program was to create a replicable model to be implemented in other schools, to 
increase the likelihood of students across the district having a successful transition to high school, graduating high 
school, and getting to and through college. The program leaders hoped to create this replicable model by learning 
directly from their experiences with this program and also from this formative evaluation to improve the program 


model, outcomes, and scalability. 


Participating Success Schools 

School Selection 

The Success Project was implemented in 10 elementary schools with middle grade students within CPS. UChicago 
Impact sought to serve high-needs schools that demonstrated they had the capacity to implement this program. 
These 10 schools were recruited and chosen from a pool of CPS neighborhood schools that met the following 


criteria: 


e Served students in sixth, seventh, and eighth grades; 


e Were in CPS networks 1-13. (The pool did not include AUSL schools since they were implementing the 


project in a different capacity); 
e Demonstrated a need for support by: 


oO Receiving a Level 2 or Level 3 rating (the two lowest ratings) from the previous CPS school 


accountability rating system; 
0 Having school-wide Title One eligibility; 


0 Having low rates of eligible students who applied to at least one high school program or selective 
high school. More specifically, schools had to be in the bottom half of all CPS elementary schools 


on this metric; and; 
oO Being in the bottom half of schools in terms of middle grades on-track rates. 


e Demonstrated capacity to implement the program by scoring “Neutral” or above on the supportive 
environment essential school support on the 5Essentials.4 If the school was a Level 3 school, then the 
school had to have an overall 5Essentials score of at least “Moderately Organized” on then districtwide 


school climate surveys. 


Ultimately, 38 elementary schools met the above criteria and were invited to participate in the Success Project; 15 
of these schools responded, from which 10 schools were finally selected. The 10 schools selected were serving the 


largest number of students in grades 6-8 among those that applied. 


School Context 

The 10 neighborhood schools that implemented this program are located in the south and west sides of the city. 
Cather and DePriest are on the west side; the rest of the schools are located on the south and southwest sides of 
Chicago. Table 1 shows the demographic characteristics of the students in these schools and the size of the middle 


grade population during Year 1 of the program’s implementation. Most schools served a large population of students 


4 The five essential supports were developed by the UChicago Consortium, in partnership with CPS. The 5Essentials survey—administered 
by UChicago Impact—provides information about a school’s climate in each of the five essential supports: Effective leaders, 
collaborative teachers, ambitious instruction, supportive environment, and involved families. For more information on the five essential 
supports, see Sebring, Allensworth, Bryk, Easton, & Luppescu (2006). For information on how 5Essentials scores are calculated, see 
http://help.5-essentials.org/ 
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from low-income families; more than 83 percent of students qualified for free or reduced-price lunch at all schools, 
with the exception of Hale. The schools varied in terms of the population of students receiving special education 
services. Some of the schools, like Claremont, served a large population of students with Individualized Educational 
Plans (IEPs) (20.5 percent) while others were below the district average. For example, less than 5 percent of the 
Wells’ students had IEPs during Year 1. Most schools served largely Black student populations. Pasteur and Hale 
had a large population of Latino students and a sizeable proportion of their students were classified as English 
Learners (ELs). 


TABLE 1 


Student Characteristics in the 10 Success Project Schools 


% Black % Latino % Students % Students % English Number of 
Students Students Receiving Free- or Receiving Learners Students 
Reduced-Price Special (Grades 6-8) 
Lunch Education 

Cather 97.5% 1.9% 97.3% 18.4% 0.5% 119 
Claremont 87.5% 10.4% 96.9% 20.5% 6.7% 151 
DePriest 96.2% 2.9% 95.3% 16.5% 1.2% 219 
Fiske 99.4% 0.6% 83.3% 8.6% 4.0% 124 
Hale 0.1% 69.3% 71.4% 13.7% 14.7% 311 
Kozminski 97.7% 1.4% 87.0% 14.8% 1.2% 107 
Nicholson 96.3% 2.6% 97.7% 11.5% 1.6% 153 
Pasteur 0.8% 93.1% 91.1% 10.0% 35.0% 382 
Wells 96.0% 2.8% 85.3% 4.7% 0.7% 136 
Wescott 95.9% 3.1% 85.6% 15.4% 1.5% 102 


Note: Data comes from publically available data from CPS web site for the 2014-15 academic year. 


Evaluation of Implementation 
The Success Project’s staff and coordinators wanted to learn as they implemented the project; they wanted an 
outside evaluator to provide them with feedback that they could take to refine and improve the project’s model. The 


UChicago Consortium agreed to provide this feedback through a formative evaluation. 


About formative evaluations: Many interventions and programs are created by organizations that attempt to 
address a need or solve a problem. Initially, these interventions are based on theory and hypotheticals; they are 
untested.> The most effective initiatives tend to learn from piloting the program, changing, or adapting it to fit the 
context or learn from experience. Individuals responsible for implementation exercise their own judgment; they use 
“human sense-making” to adapt and transform their intervention for their unique context.® These interventions are 
often more successful.? Implementation research can help program implementers understand much more about 


where they work, for whom, and under what circumstances.® 


Rossi., Lipsey, & Freeman (2004); Patton (2002). 
Spillane, Reiser, & Reimer (2002). 

Berends, Bodilly, & Kirby (2002). 

Century, & Cassata (2016). 
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Formative evaluation for the Success Project: The purpose of the UChicago Consortium’s formative evaluation 
for the Success Project was to provide information that could inform the program’s improvement as it developed. 
One objective of the evaluation was to describe how the project was being implemented in the particular context of 
the 10 Success schools and to understand the program’s theory of action.? The second objective was to provide the 


program with quantitative evidence to monitor the program’s progress. 


To understand how the program was implemented in all 10 Success schools and to understand changes within the 
theory of action over time, we conducted interviews with and observations of the Success coordinators. In the first 
two years, all coordinators were interviewed about their experiences. Additionally, we also interviewed all other 
program staff part of UChicago Impact. Moreover, in Year 1, nine of the 10 coordinators were observed in their 
schools. At the end of each year, we used this data to provide the project with information on the program’s theory 


of action as well as the challenges and successes that coordinators experienced in their work. 


To monitor the program’s progress, we used quantitative data to determine whether students’ attendance and grades 
improved while in middle grades. We also paid particular attention to the student-level survey measures from the 
CPS My Voice, My School survey that we expected the program might influence, including some student habits (like 
rigorous study habits and grit) and some school climate measures (for example, peer support for academic work and 
academic personalism). In addition, we followed eighth-grade graduates and described the high schools in which 
they enrolled and how these enrollment patterns differed from previous cohorts. The ultimate goal of the program 
was to improve outcomes for their eighth-grade graduates in high school, and to increase their probability of 
graduating from high school and attending college. But, given that only one cohort has finished one year of high 


school, we could not offer any insight into students’ performance after they left the Success schools. 


Furthermore, many studies on the implementation of education programs and reforms find that principal leadership 
proves to be an important determinant to successful program implementation.'° Principals demonstrate their buy- 
in by communicating expectations to staff and making sure resources (time, space, funds, etc.) are available; they 
support the program and want it to succeed." Having principal buy-in is a necessary early step to making a program 
effective or to lead to whole school change, but administration buy-in alone is not sufficient.'2 Educational 
implementation and education change literature suggests that even successful programs are fragile in schools. 
Programs and their progress can be undone almost overnight. This occurs in part because of administrator and staff 
turnover, but it also in part because of how successful programs are at truly embedding themselves within school 
systems and structures and influencing beliefs and school culture.’ Given the importance of principal buy-in and 
programs’ systems and structures, we paid particular attention to how coordinators approached these two issues in 


their schools. 


The remainder of the report is organized in four chapters. Chapter | contains the theory of action behind 
the program that emerged from interviews with Success Project staff and coordinators. Chapter 2 describes the 
challenges and successes of the implementation of the different areas of the program. Chapter 3 covers the 
quantitative evidence on different students’ outcomes. Chapter 4 is an interpretative summary, highlighting key 


learnings from this formative program evaluation for consideration by other programs with similar goals. 


9 To provide timely information, the research team wrote and shared with the program two memos at the end of Year 1 and Year 2. The 
findings shared in the next two chapters are distilled from those manuscripts. These documents are available upon request. 

10 Fullan (2007); Sebring & Bryk (2000); Berman & McLaughlin (1977). 

11 Fullan (2007). 

12 For example, Payne (2008). 

13> Huberman & Miles (1984). 
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CHAPTER 1 


Theory of Action 


This chapter describes the theory of action behind the Success Project, as it was described by Success Project staff, 
Success coordinators, and in initial program materials. A theory of action is an important tool that describes the 
activities of the program, the intended outcomes, and how the activities are linked to the intended outcomes. It 
articulates the ideas behind how the initiative believes it will change desired outcomes and it helps explain why a 
program achieves its intended outcomes, or does not. Through interviews in Year 1 and Year 2, program staff and 
coordinators discussed the activities and the program goals in detail and identified the underlying assumptions of 
the program. Figure 1 depicts the program’s activities and tasks, their link to levers identified by the program staff 


and coordinators, and the relationships of these levers with the outcomes that the program was looking to move. 


The Success Project intended to build three levers in schools for improving student outcomes: 1) Improving students’ 
social and cultural capital; 2) increasing attention to improving grades and attendance by school staff and students; 
and 3) getting students to apply to more high schools, and in particular, “good match and fit” high schools. These 
three levers were chosen because they have been identified in prior research as potentially important for students’ 
success in high school.'® The Success Project was organized around the idea that the combination of these three 
levers in middle school would ultimately give students a better chance to graduate from high school, graduate from 


college, and have a successful life. 


The first lever was to help students develop stronger social and cultural capital through the 6to16 curriculum.'® The 
focus of the curriculum was on five areas (College and Life Vision, Student Success Skills, Strong Student Profile, 


Social Capital, and School Awareness and Selection) and was intended to build all three levers through its content. 


The second lever was to create a focus on grades and attendance in schools. In practice, this meant educating 
students and teachers on the importance of strong grades and attendance for future outcomes, having teachers and 
administrators set goals for students to earn all As and Bs in their classes, and to attend school between 95 percent 
and 98 percent of school days. This was a higher goal than the CPS on-track indicator for grades 3-8, but was 
consistent with research that suggests students need to have 98 percent attendance and earn at least a 3.0 GPA to 


be on track for college.'7 


14 The Success Project had materials that were used to market and explain the project to external audiences, as well as internal documents 
(e.g., an initial logic model) that those working for the Success Project used to guide their work. Additionally, in both Year 1 and Year 2 
interviews, Success Project staff and coordinators discussed the program’s theory of action as it was conceptualized and as it was 
implemented, pointing to differences, changes, and challenges where present. They spoke of the tasks they were responsible for 
completing, how they carried out their tasks, and how they believed their tasks were affecting outcomes. The theory of action that we 
present in this report draws from both program materials and interview data. 

15 Alexander, Entwisle, & Kabani (2001); Neild & Balfanz (2006); Balfanz, Herzog, & Maclver (2007); BERC (2011); Kurlaender, Reardon, & 
Jackson (2008); Kieffer & Marinell (2012); Allensworth, Gwynne, Moore, & de la Torre (2014). 

16 The 6tol6 curriculum was originally developed by the University of Chicago Charter School, a unit within the University of Chicago 
Urban Education Institute (UEI). 

17. Allensworth et al. (2014); Roderick, Nagaoka Allensworth, Coca, Correa, & Stoker (2006). 
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FIGURE 1 
Success Project Theory of Action 
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The third lever was to support students in selecting and applying to “good match” (given students’ academic 
credentials) and “good fit” (given a students’ interests and preferences) high schools. This meant engaging with 
students in the high school choice process by helping them identify those high schools that could be a “good match 
and fit” for them and encouraging students to apply to more of the high schools where they were likely to be 


successful. ® 


18 This process was built off the research from Roderick, Nagaoka, Coca, Moeller, Roddie, Gilliam, & Patton (2008) where they studied 
issues around high school students enrolling in college. 
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These levers were achieved in Success schools through the 6to16 curriculum and the work the coordinators did in 
schools. The coordinators’ roles in schools focused on three tasks: 1) Teaching the 6to16 curriculum; 2) working 
with students, teachers, and school leaders to improve students’ grades and attendance; and 3) facilitating the high 


school application process. 


Since the funding for Success coordinators was intended for three years, but the goal was for the Success Project to 
be adopted by schools for ongoing use, coordinators began establishing program sustainability through the creation 
of systems, tools, and resources in Year 2. In addition to allowing for program sustainability within the 10 schools, 
these systems, tools, and resources were intended to facilitate replication of the Success Project. Often, establishing 
these sustainable structures also made coordinators’ work more efficient and effective; we elaborate on some of 


these sustainability efforts in each of the three core areas of focus below. 


Teaching the 6to16 Curriculum 

The 6tol6 curriculum was the heart of the Success Project. The coordinators and the Success Project staff who 
supported the project all talked about 6to16 as the place where all aspects of the project overlapped. One coordinator 
described 6tol16 as the “anchor of the project,” explaining that 6to16 was where all skills they hoped students would 


develop and all topics they hoped students would learn came together: 


“I think it’s the anchor of the project. I think it’s through 6tol6 that the kids learn about [Freshman] 
OnTrack....[K]ids learn about the high school piece through the 6to16. They learn about college and 
high school and the skills, you know, developing skills and practicing developing relationships with 
people. It’s all done through 6tol6...” 


The Success Project staff and coordinators intended for the curriculum to build all three levers of the project through 
its content. The 6to16 curriculum was divided into five units: College and Life Vision, Student Success Skills, Strong 
Student Profile, Social Capital, and School Awareness and Selection. The lessons in the College and Life Vision 
module were designed to help students believe that college graduation was an attainable goal. The Student Success 
Skills module aimed to help students develop skills in different areas needed to be successful. For example, one 
lesson made students aware of what clothes are appropriate for different situations (e.g., college visits and 
interviews), while another taught students how to set short- and long-term goals and have action plans to accomplish 
those goals. The lessons in building a Strong Student Profile module were intended to help students understand and 
strengthen the information that goes into college applications and what employers might look at, such as a high 
GPA, advanced coursework, recommendation letters, involvement in extracurricular activities, and so on. The work 
around building student profiles intended to improve students’ chances of achieving their academic and professional 
goals. The Social Capital module aimed to help students build support systems to help them attain their goals in 
life. For example, in one lesson, students worked to identify individuals in their lives who could help them prepare 
for college. Finally, the High School Awareness and Selection module intended to make students understand the 
necessary steps in identifying and choosing a high school that fits their interests and goals. During one of the lessons, 


students visited a high school and later reflected on what impression they had from the visit. 
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The program staff and coordinators believed that the 6to16 curriculum was greater than the sum of its parts. One 
program staff member described the curriculum “as a place where a student believes that college is possible for them.” She 
continued to explain how she thought the curriculum guided students to better short-term and, in time, long-term 


outcomes. She believed that if students know what they are interested in and what they are working toward:"9 


“...then they’re more likely to be more invested in calculating their GPA every day because they know 

how that connects to what they want later on in their lives if they know what high school they want to 

go to...and if they understand the high school application process and know what high school can do 
for them [it] can get them to the right college and can get them to the career they want...” 


The program staff initially expected 6to16 to be delivered in a whole classroom context for 135 minutes a week, but 
this goal was difficult to achieve in many schools in Year 1, given competing priorities within students’ class 
schedules. Several coordinators instead pulled small groups of students during non-core subjects. The curriculum 
was modified to help coordinators deliver the content for their unique contexts, from a short 15-minute lesson 
designed for small-group pullouts, to a longer, 90-minute block schedule with extended learning opportunities, and 
everything in between, including traditional 30-45-minute lessons. While coordinators were in charge of teaching 


the curriculum in Year 1, other school staff participated in this task in Year 2. 


The curriculum was intended to influence the students directly by making them aware of the importance of grades 
and attendance, by improving their social and cultural capital, and by helping them envision what they want their 


life to look like and think about which high school will help them achieve their goals. 


Working with Students and School Personnel on Grades and Attendance 

The second lever of the program was to work with students and school personnel on grades and attendance. 
Coordinators were tasked with increasing middle grade students’ grades and attendance by bringing both outcomes 
to the forefront of school-wide priorities and staff conversations. The coordinators’ work to improve their schools’ 
middle grades on-track rates varied substantially by coordinator and their unique school context. Coordinators took 
different approaches, including meeting with administrators, working with other key school stakeholders (e.g., 
attendance clerks), sharing data with teachers, and intervening with students. Most coordinators talked about how 
they worked with school teams or committees, like instructional leadership teams (ILTs) and grade-level teams, 
tried to build sustainable practices in their schools, and supported school-wide efforts and events around grades and 
attendance. Several coordinators discussed using tools developed by the team and/or developing tools that aimed 
to help improve grades and attendance. The ways and degree to which coordinators engaged in any of these activities 
depended on the level of administrator and teacher buy-in, including the degree to which administrators gave 
coordinators access to student-level data, committees, teachers, or students, and the ways in which administrators 


and teachers engaged with the coordinator. 


The conversations coordinators had with their administrators ranged from daily check-ins to having an established 
role to present data in ILTs—which administrators are also a part of—to periodic check-ins or going over grades 
and attendance data upon administrators’ requests. On school teams, coordinators were primarily responsible for 
sharing grades and attendance breakdowns by homeroom or grade. During team meetings, some coordinators were 
able to discuss concerns about particular students, or groups of students, and plan interventions to improve those 


students’ grades and/or attendance. Student-level interventions included check-ins with students during middle 


19 To protect research participants’ confidentiality, all individuals are referred to with female pronouns. 
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grades on-track time, one-on-one pull outs, and conversations during lunch or recess. During these times, students 
would check their grades, complete missing work, and/or discuss their goals and action steps for the week with the 


coordinator. 


The program model assumed that by focusing on grades and attendance, and by bringing this focus to staff meetings, 
coordinators could directly affect teachers’ and school leadership’s priorities. Additionally, the program model 
assumed that coordinators’ meetings with students, in which the conversation was centered on students’ own grades 
and attendance, could directly affect students’ engagement. The model also assumed that this work would indirectly 
affect all middle grades students in the building, because, according to the program’s theory of action, if teachers 
and school leaders focused on these two indicators and created systems to monitor and track students’ attendance 


and grades, then students would earn better qualifications during the middle grade years. 


Working with Students and Counselors on High School Applications 

The third lever of the program involved supporting students to identify and apply to “good match and fit” schools. 
CPS is an open-enrollment district. This means that students have a default neighborhood high school based on 
where they live, but students can also choose to apply to most of the other high schools in Chicago. High school 
options include charter schools, military schools, magnet and IB programs within a high school, selective enrollment 
high schools, and other neighborhood schools that have open seats. The deadline for the main application process 
for district-operated schools is in December of the students’ eighth-grade year. Applications to charter schools are 
governed by each individual school and have different deadlines and application processes. Further, while students 
learn which high schools have accepted them by April, the application process continues while there are still open 


seats in schools. 


Coordinators worked directly with students during the high school application process, sometimes in conjunction 
with school counselors. The goal of the Success Project was for every student to apply to at least five high schools. 
A related goal was for every qualified student to apply to selective enrollment high schools. In Year 1 of the program, 
coordinators did not enter their schools until February, which was well past many high school application deadlines, 
including deadlines for selective enrollment schools. In Year 2, coordinators were in their schools before the school 
year started, which meant they could work strategically with students and staff from the very beginning of the year 


to apply to programs and meet deadlines. 


This high school applications lever was intended to influence students directly during the application process. The 
belief was that by having students apply to high schools that were a “good match and fit,” then students would be 


more likely to attend those schools and consequently be more successful in high school and beyond. 


In sum, the Success Project was a program piloted in 10 schools that aimed to better prepare middle grade students 
for the transition to and success in high school and beyond. To achieve this goal the coordinators worked to move 
three levers in their schools: 1) Helping middle grades students develop social and cultural capital, 2) creating a 
school-wide focus on grades and attendance, and 3) supporting students in applying to and selecting “good match 
and fit” high schools. The Success Project model rested on the assumption that it was the combination of these 
three levers that would give students a better chance to graduate from high school, graduate from college, and have 


a successful life. The next chapter describes the coordinators’ experiences implementing the Success Project. 
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CHAPTER 2 


Implementation in 
Year 1 and Year 2 


This chapter describes the experiences of the Success coordinators in the first two years of the initiative, since their 
work constituted the bulk of implementation in the 10 Success schools. As mentioned in the Introduction and 
Chapter 1, each coordinator taught the 6to16 curriculum, worked with staff and students to improve students’ grades 
and attendance, helped students research their high school options, helped students complete high school 
applications, and assisted students through their high school selection process. The application requirement to 
participate in the program was intended to be a proxy indicator that the school’s staff, or at minimum the 
administrator who filled out the application, believed that improving middle grades students’ academic 
qualifications, high school knowledge, and high school applications were needs and should be priorities within their 


school. 


Through interviews in Year 1 and Year 2, coordinators discussed the program goals and their own responsibilities 
in detail, often revealing the many challenges associated with each aspect of their work as well as the successes that 
they felt and achievements that they were proud of. These reflections provided data with which to identify the 
differences between how the program was theorized and how it operated in practice. Their experiences can provide 
insight for other programs and initiatives that attempt to influence any of these outcomes. We have organized our 
findings in this chapter based on the three tasks that coordinators organized their work around: 1) Teaching and/or 
supporting the instruction of 6to16, 2) focusing on grades and attendance, and 3) working on high school application 


and selection. 


Teaching 6tol16 
Securing and Protecting 6to16 Time within Schools’ Schedule 


Prioritizing and protecting 6to16 time within school schedules was a challenge for Success coordinators 
and their schools. Most Success schools felt pressure from the district’s accountability system, which privileged 
students’ test scores. Coordinators talked about how hard it was to work in high-stakes environments, where 
instruction time was a premium and everything came second to math and reading instruction. Coordinators 
discussed how it was difficult “to convince administration that the curriculum is worth teaching in a lead-into-testing time, 
when...they cut out a lot of other instruction, like science and social studies, in favor of reading and math in order to try to boost 
test scores.” In other words, there was not enough room in the schedule for social studies or science classes, so, even 
though administrators may have signed-off on the program and wanted to create the space for 6to16 class, it was 
sometimes difficult to make a case for prioritizing 6to16 time, especially since it would have come at the expense 
of instruction in another subject. The intense accountability pressures in these schools made it difficult for 


coordinators and schools to prioritize the program and 6tol6 time. 
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Another coordinator talked about how her school was just trying to make sure that students were meeting the basic 


requirements, and that until those “basics” were met, prioritizing efforts like 6to16 would not and could happen: 


“I think often of the hierarchy of needs when you’re just in survival mode, you don’t have the luxury of 
the mental space to think contemplatively. And that’s kind of the category that 6to16 is in in people’s 
minds. It all sounds great and it would be nice for us to think about what their vision is for their future, 


but if they don’t pass math ...it’s not going to matter, they’re not going anywhere.” 


In other words, if students were performing far below grade-level in reading and math, teachers did not feel it was 
appropriate to give-up their instruction time for something like 6to16, because they believed that students needed 


basic reading and math skills to achieve any goal they may had. 


Several coordinators described successfully fitting 6to16 time into their school schedules by securing 
administrator buy-in. These coordinators prioritized changing school schedules or finding ways to work within 
their school schedules and they had administrator buy-in to do this. One coordinator described this work as needing 
to start at the beginning of the year, when planning the year’s schedule and system occurred. “Changes here start at 
the beginning of the year...[It] started with planning how can we integrate—how can we best integrate this stuff into the systems 
here so that it's not even a question of sustainability, it’s just more a question of consistency.” This coordinator also talked 
about how her school never lost 6to16 time throughout the year, not even during testing time. This coordinator was 
able to achieve this because her administration bought into the goals of the program fully and her school found 


ways to carve out the time within the school schedule to incorporate and prioritize 6to16 time. 


The exact arrangement varied from one school to another but coordinators often found that social studies, Multi- 
Tiered System of Supports (MTSS), social-emotional learning (SEL), homeroom, and/or prep classes could 
accommodate 6to16. Coordinators remarked that social studies, SEL, homeroom, and prep teachers sometimes did 
not have enough curriculum to last the year, and sometimes teachers in those classes appreciated having a 
curriculum to use during their class time. Additionally, coordinators believed 6to16 shared many goals with MTSS 
and SEL efforts in their schools. Since some schools were unable to build 6to16 into their schedules as a stand-alone 


class, incorporating 6tol16 into MTSS and SEL was a natural fit, both in program alignment and schedule. 


Creating an Accessible Curriculum 

When the program first launched, most coordinators felt that many 6to16 lessons were not accessible 
or relevant for the students in their school. In Year 1, several coordinators expressed frustration with the 
curriculum. One commonly-reported issue by coordinators was that it “went above students’ heads,” that it expected 
too much “college knowledge.” Even though the curriculum was supposed to introduce many college concepts to 
students, coordinators felt the curriculum still assumed their students knew more than they did. Many coordinators 
felt that 6to16 was not relevant to their students; they explained that their students could not imagine some of the 
scenarios that the curriculum asked them to picture. In fact, many coordinators talked about some of their students’ 
desires to have careers which students did not believe required a college education and careers which the curriculum 


did not recognize, like professional basketball players, rap artists, and YouTube stars. 


Other coordinators discussed how no one had ever asked their students what they wanted to do with their lives; 
these students had never imagined what college might really look like, so it took time for these students to have an 
idea of what they aspired to be and how college could help them achieve their aspirations. Coordinators reported 


spending a great deal of time scaffolding the curriculum to make existing lessons accessible to their students. 
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The absence of a fully developed and cohesive curriculum made teaching more difficult in both years. 
The program did not have a final version of the curriculum until the end of Year 2. During Year 1, the Success 
Project’s staff and coordinators recognized that they needed to make the curriculum more accessible and relevant 
for their students. They also learned that they would need to adapt the curriculum to be taught in a variety of 
contexts, including for small groups, traditional class periods, and longer class periods that existed in block 
scheduling. These efforts required a lot of revamping, rewriting, and reorganizing of existing material. The Success 
Project staff and coordinators hoped that they could modify and finalize the curriculum during the summer between 
Year 1 and Year 2. Unfortunately, they were unable to meet this goal and continued to work on the curriculum 


during Year 2. 


Every coordinator talked about struggling with the curriculum a bit in Year 2. Not having a complete and cohesive 
curriculum made it difficult for them to plan their courses, since they did not always know what was coming next. 
According to one coordinator, “Jt was so challenging because at the beginning of the school year it wasn’t done.” Another 
coordinator echoed this point saying, “So for me I think that that was it was just really hard because it wasn’t...done.” 
Coordinators talked about getting the curriculum piecemeal, just before they needed to start teaching it, which they 
found challenging, because they could not plan very far ahead. The same coordinator added, “Following a scope and 


sequence is really, really difficult when it’s still changing and it’s not ready.” 


At the end of Year 2, Success coordinators believed they had created a strong and flexible curriculum. 
In Year 1 interviews, we heard a lot about challenges with the curriculum; in Year 2 interviews, we heard a lot 
about how the curriculum changed and all the work it took to change it. At the end of Year 2, coordinators believed 
that 6to16 had a scope and sequence with organized units that built on one another and lessons that students could 
relate to and that teachers could adapt to fit their contexts. Coordinators were proud of the fruits of their labor. 
Nearly all the coordinators talked about re-writing or taking ownership of at least a few lessons and several of them 
talked about being a part of the curriculum redesign or curriculum revamp committee. All coordinators felt like 


6tol6 was much more accessible as a result of the revamping efforts in Year 2. As one coordinator put it: 


“So, [the curriculum last year] was advanced. ... There was a certain level of assumption that the 
students knew what it takes or knew what this meaning was. And I think this year, the material is so 
broken down to the point that they can understand it and they can relate to you, and so you have more 


engagement, because it’s more relatable.” 


Some coordinators also believed that the curriculum would need continued updating over time to stay relevant and 


to keep up with current events. 
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Not only was the updated curriculum more accessible for students, it was more accessible to school staff. 
Coordinators talked about how the updated curriculum looked like a curriculum, with a scope and sequence and 
units with approximately three or four lessons on each topic. Coordinators talked about how they designed the 


curriculum to be flexible to meet the needs of different school contexts. In the words of one coordinator: 


“There’s different components to it so that it becomes very flexible for people to read. So, if a teacher 
only has 15 minutes, there’s a mini lesson that they can teach in those 15 minutes. If a teacher has an 
hour, there’s five components that they can teach that has an introduction, a connection to real-world— 
like, why you need to learn this. There’s a mini lesson. There’s a student-engagement piece. There’s a 


connection to core subject...if it applies. So it’s flexible in that way.” 


Coordinators expressed how proud they were of the work they did to create this curriculum and how excited they 
were to use it: “So it’s very structured and organized, which is great and when I saw the rebranding I was like ‘Oh 


By) 


my gosh, I’m actually excited about this now. 


Learning to Manage a Classroom 

Classroom management issues sometimes prevented coordinators from delivering content and engaging 
students. Several coordinators reported struggling to manage their 6to16 classroom. Part of these struggles had to 
do with some coordinators’ lack of previous classroom management experience. Many coordinators believed that 
their schools had a chaotic school environment and lacked school-wide systems, strategies, and practices to manage 
student behavior. These management issues—whether stemming from coordinators’ lack of experience or from a 
disorderly school climate—prevented coordinators from getting through their teaching points and students from 
really engaging with the material. As one coordinator explained, “Buy-in, engagement, behavior, I mean I think it’s all 


tied together.” 


Classroom management struggles had repercussions for the Success Project and the coordinators’ 
relationships in schools. For instance, one coordinator talked about how struggling with classroom management 
undermined her efforts to cultivate teacher buy-in, saying, “Classroom management was a problem and buy-in from the 
teachers was just sort of like, ‘I'm losing my instructional time so that kids can disrespect you. Doesn’t seem worth it.’” These 


teachers viewed 6tol6 as a waste of time, which undermined the coordinators’ efforts beyond 6to16. 


Several coordinators found that teaching small groups was an effective way to deliver 6to16 content 
and solve classroom management challenges. One coordinator specified, “For me it was just a lot better in terms 
of being able to just not be overwhelmed.” Though some coordinators taught small groups not by choice but by necessity 
(because they were not able to have programmed, scheduled time within the school day), most coordinators were 
able to make this set-up work, even if it required heavily modifying the curriculum to shorter 15-30 minute lessons 
with a small group of students. Even coordinators in school environments with tremendous accountability pressures 
were able to pull students for short periods of time to deliver instruction. Further, sometimes coordinators found 
that they could really engage students with more tailored, near one-on-one discussions around goal setting, 
organization skills, grades and attendance, and their long-term vision. As we will discuss later, the coordinators 
also found pulling small groups of students to be an effective way to do high school application and selection work 


with their eighth-grade students. 
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Coordinators also sought experts to coach them around classroom management and instruction issues; 
they found these individuals to be very helpful. Coordinators craved professional development around their 
instruction and how to set up their classroom environments from experts. These experts included individuals within 
UEI who were unaffiliated with the Success Project—staff who train teacher candidates and coach new teachers 
within the University of Chicago Urban Teacher Education Program or staff who once coached teachers in a 
position they held before joining UEI—and consultants hired by UChicago Impact to provide their coordinators 
with requested support. A couple of coordinators also talked about reaching out to teachers in their schools for 


support to improve their classroom management practices. 


In both years, several coordinators talked about how their instruction improved as a result of the efforts the program 
made to improve their classroom management. In Year 2, one coordinator reflected, “I guess I realized how bad I was 
doing, so I started inviting people to come in and give me feedback on how my class was going. Trying to be deliberate about 
incorporating their feedback into my class has really worked.” Another coordinator described how her classroom changed 
once she received some help with behavior management and after she spent a couple of weeks trying-out new 


strategies in Year 1: 


“...So I started trying new techniques, and I started seeing little changes here and there. And then, 
after a couple weeks, I was a lot more successful than I was and I could actually get through a full 
lesson and not have challenges and students were actually asking questions about what we were talking 


about in class, and it was a lot better and a lot more positive than the earlier classes have been.” 


By the end of Year 1, this coordinator felt that improving her management skills was “one of [her] successes,” but the 
coordinator needed some support and time to improve. Once she started improving her classroom management 


practices, she felt like her overall instruction, and her levels of student engagement improved as well. 


Grades and Attendance 

School Priorities and District Accountability 

Coordinators felt that test-based accountability made it difficult for schools to prioritize improving 
student grades. The fact that schools are evaluated mainly by test scores posed a barrier for coordinators’ work 
on improving students’ grades and attendance. The schools that coordinators worked in were low-performing 
schools that faced pressure from the district’s accountability system. CPS’s accountability system includes a number 
of student metrics, such as attendance, but to a lesser degree than it weighs student test scores. Consequently, though 
the Success Project privileges grades and attendance, coordinators worked in school environments where students’ 
test scores mattered more. They described how their schools felt pressure from the district to improve their test 
scores; some even talked about teachers fearing that their school would be closed or that they would be fired because 


of low standardized test scores. One coordinator offered some insight into this, saying: 


“And test scores are a big deal. Teachers, I think, are a little scared that they'll get fired if they don’t 
receive a certain amount of growth on the NWEA from year to year...the NWEA is the most important 
[metric] by far, and that’s the one that gets stressed because it’s included on the school report card. But 
the other ones, —they haven’t been enforced as strongly as possible. And as a consequence, they haven’t 


necessarily been where they need to be. 
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The coordinator in the above quote not only described why test scores matter, but she went so far as to say that “the 
other ones,” meaning the other middle grades outcomes like grades and attendance, were not given the same priority 
as test scores. Additionally, the coordinators talked about numerous instances where their work took a backseat to 


the standardized testing schedule and to core course time that prepared students for standardized tests. 


Working to improve attendance was easier than working to improve grades. Attendance is also a part of a 
school’s accountability rating; therefore, many of the Success schools were already working to improve attendance 
before the coordinators entered their schools. Coordinators said they could capitalize on this momentum and 


contribute to already-existing school-wide attendance systems efforts in both Year 1 and Year 2 interviews. 


Several coordinators discussed their school-wide initiatives to improve attendance. These school-wide efforts 
included offering incentives for students above a certain attendance rate (from popsicles at lunch to bike raffles), 
clearly posting daily or weekly school-wide attendance for students, staff, and parents to see, and awarding an 
attendance trophy to the class with the highest attendance. In this last example, the attendance trophy traveled 
around the middle school to the class with the highest attendance. When coordinators were in schools with these 
systems already in place, they discussed their role as just working to support or bolster what was already in place, 


adding what they could. In these schools, attendance was a shared priority between the program and the school. 


In some instances, especially in Year 2, some coordinators discussed some of these school-wide incentives focusing 
on middle grades on-track rates. In other words, coordinators were sometimes able to turn school-wide attendance 
efforts and systems into efforts and systems to improve both attendance and grades. Some of these school-wide 
initiatives included events like on-track ribbon pinning ceremonies and using being on track as a criteria to gain 
certain privileges, like open gym time. Coordinators talked about on-track and attendance efforts successfully 
encouraging students to bring-up their grades and come to school; they heard students internalize the importance 
of on-track. Coordinators shared that after their students learned how to calculate their GPA and attendance during 
6tol6, they would excitedly share their on-track status with coordinators and that their students wanted the 
recognition and benefits that came with being on-track in their schools. Some coordinators talked about student 


ownership of their grades and attendance as a major success of their time in schools. 


The district’s increasing recognition of third- to eighth-grade on-track encouraged schools to pay closer 
attention to grades and attendance data. Though third- to eight-grade on-track rates was not a part of the 
accountability system, CPS’s increased focus on on-track data led some schools to pay closer attention to grades in 
Year 2, and coordinators helped their schools keep track of grades data. Some coordinators talked about their 
principals receiving pressure from their network chief to improve third- to eighth-grade on-track rates during Year 
2 of the Success Project. In the past, network chiefs put pressure only on improving student attendance. 
Consequently, some administrators approached coordinators with questions about their grades data in a way that 
did not happen in Year 1. One coordinator described how network pressure drives the questions her principal asks 
of her: 


“(The administrators] usually have a lot of questions, depending on what they hear from the network in 
that particular day or week...like, ‘Where are we in this metric?’ Or, ‘We need to—for this measure for 
SQRP, where are we? What can we do for these diverse learners?’ That kind of thing. ... There’s a lot of 


being reactive to what’s coming down from above.” 
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Working Towards As and Bs 

Improving students’ grades, especially pushing them to earn As and Bs, was sometimes tense work. 
Improving students’ middle grades on-track rates was a stated goal of the project but, pulling from research on the 
transition to high school, the Success Project also hoped to push more students to earn As and Bs. This work proved 
to be tricky, since it revolved around teachers’ grading practices and students’ content knowledge. One coordinator 
stated that there was “a general expectation that [students] should be doing better than they do” in her school. Coordinators 
had a hard time with this—they did not want teachers to give unearned grades, but they also felt that grading 
practices were more conducive to students achieving As and Bs in some other schools. Several coordinators 
explained that some teachers did not believe that low grades were about students missing a few assignments or 
students just missing a few points here and there across several assignments; coordinators said that some teachers 
did not believe low grades were even about student effort. Rather, coordinators discussed how several teachers 
believed that some students’ skills and knowledge were just not where they needed to be, that these students could 
not earn good grades, based on the content and skills that they had and had not mastered. From their vantage point, 
students have to produce high-quality work to earn high grades, and if students did not or could produce high- 
quality work, then teachers felt that they could not give students the grades that coordinators pushed for (Bs or 


better). These challenges and realities often made it difficult for students to earn As and Bs from their teachers. 


Importance of Relationships and Trust 

Coordinators believed that building and cultivating relationships with administrators, teachers, and 
students was a fundamental part of their work, especially around grades. Coordinators felt that relationship- 
building with teachers and other staff was necessary in order to do the real work of the program, that building trust 
was “needed to really get at affecting the change that we’re looking to affect.” This same coordinator felt that “there needed 
to be time spent to build the relationships...and I think that is still underway.” Further, as this coordinator alluded to, 


building relationships takes time. It took time to establish trust, relationships, and buy-in. 


In Year 1, some coordinators discussed struggling to build relationships with students at the beginning of the year; 
they said that students questioned their authority and commitment. Coordinators also shared how they were able 
to win students’ buy-in over the course of several months, which often meant going above and beyond their assigned 
responsibilities to demonstrate caring and commitment to their students (e.g., tutoring students after school, having 
lunch with students, respecting students’ dignity by meeting with students one-on-one outside of class to discuss 
students’ misbehavior, etc.). These coordinators believed that students first needed to buy-into them, the 


coordinator, before they could buy-into the work the coordinator was trying to do. 


Additionally, in Year 1, coordinators struggled to find the time and strategies to engage teachers and other school 
staff in deep conversations around the Success Project and/or middle grades on-track rates. At the same time, 
coordinators discussed this first year as a relationship-building year and talked at length about the work they were 
doing to build relationships and plant the seeds for future years. Building rapport often involved going above and 
beyond their job responsibilities, including lunchroom and recess duty, subbing or filling-in for teachers when 
necessary, sitting on school committees, running study hall, working on the school discipline policy, and sometimes 
even counseling or doing behavior interventions with students. Some coordinators noted that all other school staff 
would be responsible for many of these tasks, in addition to their own teaching responsibilities, as well. One 


coordinator described the efforts she took to build these relationships: 
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“...in order to build capital with teachers, [I] am [working] outside of my job description, being a 
chaperone on a field trip, being the person to fill in to the pre-K room when we don’t have a substitute 
teacher...essentially, any and everything that I can to try to build those relationships that I eventually 


leverage to be moving the needle on the goals that we have for the Success Project.” 


Coordinators believed that they could not ask teachers and other school staff to participate in the project without 
being seen as team players, hard workers, and people who cared about their students and school community. They 
believed that in order for teachers and staff to engage in conversations about the Success Project, to ask them to 
take-on Success Project-related work, or to ask them to reflect on their grading practices, they had to have a positive 


working relationship built on trust. 


Administrator buy-in was critical for middle grades on-track improvement. Some coordinators remarked 
that they were only able to make the impact they had because they had buy-in from their administrators. One 
coordinator offered, “T think it just is very rare that you work in an environment where people are like ‘Whatever you say I 
believe that you have good intentions and it’s gonna help our kids so do it.’ ... There are no limits and I think that that has been 


a really positive thing.” 


Given the role of the coordinators in schools, most changes to teacher practices came through 
coordinator’s influence, not their authority. Coordinators were in a tenuous position; they were outsiders. 
Though they worked in their schools most of the time, they were not CPS employees, they had temporary positions, 
and still, they were tasked with asking teachers to change their practices. Since the coordinators did not have any 
authority in their schools, this proved to be delicate work. As one coordinator stipulated, “We’re not their boss, so we 
can’t tell them what to do.” Coordinators talked about trying to find small entry points saying things like, “7’l/ kind of 
slide in, ‘Well, you know they’re having, you know, this concern, and you know, is there a way that you feel like we could 
support him better?’ you know, just having like that sidebar over lunch.” Another coordinator talked about looking for 
small entry points and providing continuous gentle nudges. She described a situation where she was meeting with 
teachers and going over their data and the she discovered that there were three students with Fs. She then asked the 
teachers, “Is there anything that we can do to support—like is it just home? Is it here? Sort of what’s going on?" The 
coordinator added that the “conversation usually goes from that in terms of like, ‘Well, I thought they did—I thought they 
knew it. They did really poorly on the test.’ And it’s just sort of like, ‘Well, did you give them a chance to retake?’ ‘No.’ ‘Do you 
want to?’ ‘Probably.’" The coordinator went on to say that this simple approach worked with several teachers for 
some students, “but it doesn’t work with everyone.” So, while coordinators discussed some wins getting teachers to 
examine their data, give students more opportunities, and rethink their practice, there were occasions where this 
approach did not work, and, again, coordinators did not have authority to do much more than this. This often left 
coordinators feeling stuck. In order for teachers to be vulnerable enough to have these delicate conversations with 


their coordinator, coordinators needed to establish a level of trust with their teachers. As one coordinator explained: 


“T also feel like with [middle grades on-track] really took that first year to gain the trust because I’m 
looking at teacher’s grades books and I’m looking at things that really had the potential, I think, for 
people to just feel really vulnerable about and kind of suspect like how are you going to use this? Am I 


going to get in trouble? Are you on my side?” 
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Coordinators thought that trust was essential between students and teachers to engage in conversations 
around improving students’ grades. The Success Project’s theory of action included students building self- 
advocacy skills as part of building social capital. A huge piece of that self-advocacy was students going to their 
teachers to ask questions about a grade, to request that teachers change incorrectly entered grades, and to work with 
teachers around missing work or re-takes. In some schools and with some teachers this was challenging, especially 
when, from coordinators’ perspectives, there was not a lot of established trust between the teacher and student. 
Coordinators shared that their students said, “that the teachers are not giving them chances. And the teachers [said] they 
do give them chances and they’re not taking advantage of the chances. So it’s kind of been like a seesaw back and forth.” Several 
coordinators talked about teachers and students blaming each other when it came to grades. Coordinators said that 
in order to create buy-in with teachers and to cultivate a willingness to have some delicate conversations with 
students, coordinators worked with teachers: “We have to stop this kind of back-and-forth like it’s the kids’ fault. Then the 
kids blame the teachers. We have to let it go and just humble ourselves and say, okay, what do we need to do to get these kids 
where they need to be and figure out ways to work around that!” Coordinators talked about these scenarios as a “blame 
game,” where the student blames the teacher for giving a bad grade and the teacher blames the student for earning 
a bad grade, without either party taking responsibility or being open to hearing what the other party had to say. 
According to coordinators, these conversations between teachers and students often left students and teachers 


feeling disrespected. 


To help some students have productive conversations with their teachers, some coordinators provided their students 
with scripts. One coordinator talked about a script she provided to her students to have a respectful conversation 
when they asked their teacher about their grades. Coordinators reported working with their students on their tone, 
because students did not necessarily understand how their tone might influence a teacher. “I’ve had students say, 
‘Well, so-and-so doesn’t respect me...why can’t I just talk any way to them, when they don’t talk respectfully to me?’” Or 
students did not want to talk to their teachers, because they never felt respected. One coordinator talked about 
students who said things like, “‘It never goes well’ or ‘You don’t know what this teacher is like.’” The coordinator added, 
“And so that’s been hard because [respect]’s not shared” between students and teachers. In light of these challenges, 
coordinators reported that students and teachers found scripts useful. One coordinator commented that a teacher 
approached her to go over a conversation together, because the teacher recognized how successful the coordinator’s 
conversations with students had been and how much the teacher struggled to have these conversations with the 
students. The teacher hoped that she would get better at having conversations with students about their grades with 
some extra support. Additionally, students were learning strategies for successful self-advocacy that they could 


apply in future situations. 


High School Application and Selection 

Buy-in from School Counselors and Students’ Families 

Counselors had competing priorities, thus coordinators often led most of the high school application 
process. Working on high school application and selection often involved working with the school counselor. 
Coordinators stated that their counselors were overloaded with responsibilities. In some schools, coordinators said 
their counselors were solely responsible for IEP implementations, behavior interventions, testing, and high school 
application work. Coordinators often struggled to find time to meet with their school counselors around student- 
level interventions and outcomes, which many coordinators attributed this heavy workload. When coordinators did 
work with counselors around high school application and selection, coordinators often supported the existing 
systems and structures the counselors had in place for high school applications. However, most coordinators talked 
about taking this work over, and in some cases even working independently of their school counselor. In some of 


these cases, coordinators faced the challenge of building up an application system from scratch. 
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Counselors were critical partners for high school applications. Even though coordinators talked about largely 
leading the high school application process, they also mentioned there were aspects of this work that only counselors 
could do, like sending student transcripts. They described having all of the responsibility without the ability to 
control each piece they needed to see the process through completion. In the words of one coordinator, “Owning the 
entire process was really, really, really, really hard.” The coordinator added that “it’s really frustrating” when students, 
staff, parents, or the program get frustrated “about things that happen that are not in the locus of your control.” This 
coordinator gave an example of several students she believed could have attended a particular school but who did 
not get into that school because the counselor did not fax their student transcripts and complete the application. 
This coordinator was incredibly frustrated, because she did all she could to complete students’ applications, but she 
did not have the authority to fax her students’ records and finalize their applications. Counselor buy-in and 


engagement in this process was an absolute necessity to ensure that students’ applications were completed. 


Parents were also an essential player in this process, and getting their support was sometimes 
challenging. Coordinators spoke about how critical parent buy-in and support were to the high school application 
process. For many, getting parental support in terms of getting parents to sign proxy letters was a critical step. 
Having these proxy letters allowed schools (e.g., coordinators and/or counselors) to fill-out and submit high school 
applications without getting individual parent signature on each application. Several coordinators talked about 
organizing parent meetings or using school-organized parent meetings to walk parents through the whole 
application process. Some coordinators shared their students’ grades and test score data with parents in these 
meetings, which allowed parents to have an idea about some of the options that were and were not available for 
their students. Coordinators and counselors often passed out and collected proxy letters at those meetings. In some 
other schools where coordinators struggled with parent engagement—including low parent turnout at evening 
meetings—coordinators talked about making themselves available to meet with parents whenever and wherever 


parents were available. In the words of one coordinator: 


“It’s really hard to have regular sessions with parents, especially when a lot of them work non- 
traditional schedules or generally aren’t engaged in the school community. So I mean, I think every 
education, non-profit, what have you, is trying to solve parent engagement, and I certainly haven’t 


solved it, but my way of doing my best has just been to say, ‘When you’re free, I will talk to you.’” 


Other challenges with parent engagement were the varied expectations and desires that parents had for 
coordinators’ involvement. Some coordinators struggled when parents wanted coordinators to do the whole 
application process on their student’s behalf without the parent’s participation. Coordinators believed this occurred 
when parents did not know how to engage in the high school application process and instead placed their trust in 
the coordinator to do what would be best for their child. On the other hand, coordinators also faced challenges 
when some parents did not want coordinators involved in the process whatsoever. In these instances, coordinators 
were not allowed to work with students or counsel them through the application process at all. Coordinators found 
this frustrating for two reasons: first, they could not help students or offer students any advice even when a student 
asked them for help; second, they were still held accountable by UChicago Impact for students completing at least 


five applications. 
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Identifying a “Good Match and Fit” High School 

Finding a “good match and fit” high school was an individual process for every student. Because students’ 
qualifications, needs, goals, and limitations were unique to each student, the process for finding a high school that 
was a “good match and fit” was a unique process for each student. The Success coordinators had an easier time 
identifying “match” high schools for their student, since “match” was based on a student’s qualifications and a 
school’s admissions requirements. They had a harder time defining high school “fit,” since there are so many factors 
that could make the school a good option for a student and contribute to a student’s academic and non-academic 


ta 


success. One coordinator succinctly defined fit as “...it’s almost like your shoe or your glove, like it just feels right you 
know when you put it on. When you go there you fit in,” in other words part of fit is a school’s environment and a 


student’s sense of belonging. 


The curriculum also initially identified fit as including aspects like the programs, classes, and activities that schools 
offer for students and if those matched what a student was looking for. Another coordinator offered an example of 
what fit looked like for a student who played and prioritized basketball, “Z talk about what a school has that will help 
them be successful. So that can be a number of different things. So for...students who want to be athletes and who play sports 


and who are like on AAU teams, that is an element of fit. But it’s not an element of fit for everyone.” 


But the program staff and coordinators learned that this definition of fit was not comprehensive enough; they needed 
to expand their definition to include aspects that the program had not previously considered. The same coordinator 
as above continued to expand the definition of fit by saying a good fit high school is one that will support students 
and meet their own unique needs and realities. These needs and realities often include a student’s commute to and 
from school and/or the school’s proximity to the student’s home and families’ needs and concerns. The coordinator 
offered “So a lot of them are older siblings, so they can’t realistically go more than 20 minutes away, because they have to drop- 
off or pick-up little brothers or sisters. So it’s fitting all of those things, their interests into what sort of allowance they have in 
their world.” In this vein, most coordinators discussed that they did not know what school was best for students, 
that they could not make this call for students and families. One coordinator explained, “I can’t determine what works 
for a family. And if a mom doesn’t want her son to ride the Green Line [a Chicago train line] at 6:00 in the morning, 5:00 in 


the morning, then I’m not going to tell her that that’s her best option. Because it’s not. If she doesn’t think it is, then it’s not.” 


Families’ daily considerations and personal histories with schools placed limitations on students’ high 
school options. Many coordinators discussed how the challenges of families’ daily considerations manifested in 
high school selection: Parents often prioritized safety, which, in most cases, limited students’ high school options 
to those that were in close geographic proximity. This left students and families, as well as the coordinator, “between 
a rock and a hard place” as the schools that were close by were often of poor academic quality or had admissions 
standards that students did not meet. One coordinator discussed this very difficult reality. She worked in an 
elementary school that had very few high schools in close proximity and the parents she worked with would not 
consider the closest selective enrollment school because they “/were] not fond of the community” and most of the 
students she worked with were ineligible to attend one of the other few nearby options. This coordinator explained, 
“A lot of our students don’t get in because it’s a magnet program.” Other schools did not have high schools without 
academic admissions criteria nearby, such as charter high schools. “So it put us in a very difficult place, because anything 
else [beyond the local neighborhood high school], they’re traveling... So they are very limited on options compared to other 
schools who have like charter networks and that, so we don’t have any charter schools close by.” This coordinator’s students’ 
options were limited by geography, concerns of safety, their own academic qualifications, and the quality of nearby 


schools. 
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Sometimes students wanted to apply to or attend schools that their parents would not consider. Coordinators talked 
at length about how excited most of their students were to engage in the high school application and selection 
process—to think about what they wanted their futures to look like and what schools would help them achieve their 
goals. Moreover, coordinators felt that their students trusted their advice around the process. However, parents did 
not always agree with students’ preferences. To illustrate this point, one coordinator gave the example of a student 
who really wanted to be a doctor and the coordinator, the counselor, and the student all believed that Crane Medical 
Prep would be an excellent fit for her. The school has a medical focus and partners with Rush University to give 
students paid internships at the hospital where they wear white coats and go on different rotations with doctors and 
where they can take community college classes to earn some credits in the later years of high school. For these 
reasons, Crane was an attractive option, but the student’s mother would not consider letting her attend the school 
because of her past history with the school. “Mom came in and she was like ‘Oh, she’s actually not gonna apply to Crane.’ 
...She’s like ‘Oh my god, Crane is such a terrible school, da, da, da.’ We’re like ‘Yeah, it used to be, absolutely, but they phased 
all of the kids out who were in the other program and now it is a medical school.’” The coordinator and counselor explained 
the way the school currently operates and all it could offer the child, but the parent would not sign-off on the school. 
The coordinator believed, “The mom had this idea of what it used to be and, and where it used to come from and the, you 
know, stories that she heard and the friends that she had that went there. And so I think that that was one of the biggest issues, 
kind of dispelling the myths of what schools used to be.” 


Coordinators expressed a desire for the program to have “realistic expectations” for the number and kinds of high 
schools to which their students should apply. The broader definition of fit, one that heavily considers students and 
their families’ preferences and concerns, constrained the schools that students could consider. One coordinator 
expressed a desire for the program to be more understanding of the constraints students and families are in when it 
comes to high school choice and to have more realistic expectations for what the coordinators can do in terms of 


high school choice. She said: 


“That’s not being realistic. I understand we want more kids to apply to selective enrollment 
schools...selective enrollment may not be the best for every kid. Realistic about certain things, like, 
when it comes to high school applications especially, for a lot of parents, small things like safety 
transportation plays a role in how far the kid can actually go... I don’t know if it’s putting themselves 
in the shoes of the parents or the shoes of the community... But just really being realistic about the 


situation we’re in, the schools we’re in...” 


The Workload 

The high school application and selection process was a yearlong effort. The program model was initially 
built with the assumption that high school selection work would end once applications were submitted in December. 
Once the Success coordinators started doing the high school application and selection work, however, they and the 
rest of the Success Project staff learned that the work continued long after December. Students needed to research 
their options and to fill-out multiple applications for multiple schools and different kinds of options, which had 
many different deadlines. Parents needed to be informed about the application process, to consent to coordinators 
helping their child, and to sign-off on applications. Not all students were accepted to schools they initially applied 
to so coordinators helped those students fill-out more applications. Students and families often then needed support 


to think through their options. 
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Doing all of this work was a heavy lift and coordinators described it as a truly yearlong process that started at the 
beginning of the year and ended late in the spring. In the words of one coordinator, “Really the high school application 
season, um, peak is really October to December...But then there are acceptances. That process for students to be ready to 
accept...is February and March.” The coordinator also added “then any students who still aren’t placed, another window 
opens up in May” and clarified “So it really is year-round to make sure that every student is placed in a quality high school 


that’s the best fit for them.” 


To make the time for high school application and selection work, coordinators used 6to16 time with students to do 
high school research, having students look at school websites to find graduation and Freshman OnTrack rates, as 
well as the kinds of activities and programs offered so they could find schools that were a “good match and fit.” 
Coordinators also used 6to16 time to fill out some high school applications together. Coordinators often pulled 
small groups of students based on their qualifications to fill out other applications together. For example, a 
coordinator would pull students who were eligible to apply to selective enrollment schools or schools with IB 
programs and fill those out together; they would pull students who were eligible to apply to military academies and 


fill those out together. 


Coordinators also did a lot of this work outside of their 6to16 time. Some coordinators talked about hosting high 
school fairs or high school application boot camps after school, where they had a folder for every eighth-grade 
student with all of the information they would need, including their user name, password, test scores, and grades. 
Coordinators talked about taking over computer labs or having students crammed in their offices trying to get all 
their applications out the door. Some coordinators also tried to collaborate with middle grades teachers in their 


building, like the English teacher, to have students write application essays in their class. 


Perhaps the greatest challenge of the high school application process was that it took an incredible 
amount of time and energy. High school application and selection was a process that coordinators almost entirely 
owned, largely because they felt no one else in their schools had the capacity to own the process, as discussed above. 
One coordinator who talked about not being able to work with her counselor because of her counselor’s competing 
responsibilities said, “I basically did the whole thing.” And added that when she talked to her counselor about this, 
her counselor told her, “‘No, basically you are doing this on your own. I don’t have time for this.’” Six coordinators 
reported that they almost entirely owned this process with minimal involvement from their school counselors. 
Another coordinator commented that her counselor was “swamped, but tried to be involved when she could.” Three 
coordinators talked about sharing the responsibilities and working collaboratively with their counselors. One 
coordinator who did work closely with her counselor said that she didn’t know how she could have done it all on 
her own, because “It was so much work. It was... really insane. It was really a lot of work and I just can’t imagine how you'd 


do that with just one person.” 


Though coordinators spent a great deal of time and energy—more than they expected to—on the high school 
application and selection process, and despite the many challenges that coordinators experienced within this bucket 
of work, this was also where they expressed having the biggest impact and the most success. Coordinators described 
leaping for joy and running or skipping down the halls when their students got into selective enrollment high 
schools. Some coordinators helped their school see their first students accepted to schools like Lane Tech or 
Whitney Young, two highly competitive selective enrollment high schools. Coordinators celebrated and took joy in 
their students’ acceptances as an important and necessary step for those students to achieve their other academic 
and career goals. For some coordinators, these acceptances came after months of trying to convince students that 


they could get into these schools. For example, one coordinator said: 
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“T actually had one student...when he pulled out a stack of letters, I said, ‘I told you so.’ And he said ‘T 
didn’t believe you. You kept telling me people wanted me, I didn’t think anybody wanted me.’ I was 
like, ‘I told you, look at all those people.’ And he was just beaming.” 


In addition to celebrating individual student acceptances to selective enrollment schools, coordinators were also 
proud of how students have internalized the message of finding a “good match and fit” high school. When asked 
about her biggest successes in Year 2, one coordinator talked about all of her eighth-grade students, saying, “They 
all are placed in high-quality schools that are a true fit for them, and they are able to articulate their reasons for going as being 


tied to Freshman OnTrack, or graduation rates, or college enrollment rates, and their futures.” 


Through their implementation efforts, the Success Project staff and coordinators learned a number of lessons that 
spanned the work in all three levers, especially the importance of securing buy-in and trust, and the pressures of 
district accountability. Securing buy-in from administrators helped protect 6to16 time and made easier to work on 
other areas; establishing trusting relationships with teachers and students was important for all aspects of their work 
and especially critical for the work on grades; and, coordinators learned that the high school application and 
selection work hinged upon building good rapport with counselors and families. Test-based accountability pressures 
affected the extent to which 6to16 could be taught and when; accountability pressures also affected which student 
outcomes were school priorities. Last, the particularities of school contexts influenced attendance and grades 
interventions, the length, delivery, and content of lessons, and the high schools that were in close proximity, which 


had implications for the high schools to which students applied. 


Despite all the challenges the coordinators faced, they reported successes in all three areas. Coordinators were proud 
of the 6to16 curriculum that they created, which was adaptable to different contexts and which their students found 
relatable and engaging. Another big success for coordinators was bringing students and teachers together to work 
on improving students’ grades. Coordinators were pleased when they saw teachers and students having productive 
conversations with each other around students’ grades and when they believed these conversations helped students 
and teachers build deeper relationships with each other. Perhaps the most rewarding aspect of coordinators’ work 
were the results of their efforts around high school application and selection. Coordinators took great pride when 
their students articulated why they chose a particular school and when students received admittance to programs 


or high schools they were excited about. 
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CHAPTER 3 


Middle Grades Outcomes and 
High School Selection in the First 
Two Years 


As previously mentioned, the Success Project sought to better prepare middle grades students for the transition to 
high school and to succeed in high school and beyond. In the short-term, the goal of the program was to improve 
student outcomes for middle grades students. In particular, the focus was on raising middle grades on-track rates 
by improving grades and attendance. Another goal was for students to engage in the process of high school choice 
and to increase the percent of eighth-graders who applied to selective enrollment and other “good match and fit” 
high school options. In the medium-term, the Success Project expected that students would have a better transition 
to high school and a higher likelihood of graduating high school and attending college, because students would 
attend high schools that matched their qualifications and fit their needs and academic goals. By improving these 
outcomes, in the long term, the program expected to produce higher college graduation rates for students who 


attended Success schools and were a part of the Success Project. 


Given the short time period the Success Project piloted in schools, we are limited to exploring two different sets of 
outcomes in our formative evaluation. First, we will look at students’ middle grades outcomes, including GPA, 
attendance, and middle grades on-track rates. Second, we will follow eighth-grade graduates and describe the high 
school choices they made. The purpose of these analyses was to give initial evidence of effectiveness of the program 
on improving key student outcomes and not causal evidence. Eventually, the program would expect greater impact 
on outcomes for their eighth-grade graduates in high school, but given that only one cohort has finished one year 
of high school and that cohort only received a half of the a year of the program, we cannot offer any insights or 


conclusions beyond those shared in this chapter. 


Middle Grades Outcomes 

The intervention targeted students in sixth, seventh, and eighth grade. Given the goals of the program, we explore 
whether there were any changes to students’ attendance, grades, and middle grades on-track rates. CPS defines a 
student as being on-track in grades 3-8 if they have 95 percent attendance and a C or better in reading and math 
courses. In addition to considering reading and math courses, which are the center of the middle grades on-track 
definition, we investigate changes in core GPA, which is the combination of reading, math, science, and social 


studies GPAs. Prior work has shown that core GPA is predictive of students’ performance in ninth grade.?° 


Besides these student-level key outcomes, our analyses includes a few survey measures based on students’ responses 
to the annual CPS My Voice, My School survey. This survey collects information of a wide range of topics including 
school climate, classroom environment, academic rigor, and students’ experiences and feelings, such as the support 


they receive from their teachers, how safe they feel at school, and their schoolwork. We analyze survey measures 


20 Balfanz et al. (2007); Allensworth et al. (2014). 
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focusing on two particular areas: Behaviors and strategies that the intervention might have affected (such as rigorous 
study habits and grit) and also perceptions of students about their peers, teachers, and school (peer support for 
academic work, academic personalism, student-teacher trust, and school connectedness). Because the intervention 
pushed students and school staff to pay more attention to grades and highlighted the importance of having better 
grades, students may have put more effort into their classes and worked harder. As coordinators reported, there 
were also more conversations taking place between students and teachers about their schoolwork and this might 
have changed the perceptions that students had of teachers and their school (see Appendix A for a detailed 


description of the survey measures). 
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How We Conducted the Analysis of Student Outcomes 


and How to Read the Graphs 


This chapter presents a series of analyses measuring the changes in student outcomes in the 10 schools where 
the Success Project was piloted before and after implementation. The years before implementation include 2011- 
12, 2012-13, and 2013-14. The years of implementation include 2014-15 (Year 1; partial year) and 2015-16 (Year 2). 
The analyses made adjustments for the background characteristics of students in the schools so that the effects 
were net of any changes in the student population in the schools. 


Schools included in the data: In addition to the changes observed, we provide the same analyses for other 
groups of schools so that readers of this report can gauge whether these changes were similar to what other 
schools experienced in those years or whether the changes were higher or lower compared to other schools. We 
offer three other groups of schools for comparison: 


e The first group consists of all the AUSL elementary schools. These schools implemented a model with 
some elements similar to the Success Project. In the year 2014-15, 23 AUSL schools were part of this 
group and those are the schools included in the analyses. Four more AUSL schools implemented this 
program in 2015-16, but they are not included in the analyses because they started a year later and three 
of them became turnaround schools the year prior.” 


e The second comparison group includes the group of schools that were invited to apply to the Success 
Project. The comparison group is formed by 28 schools. These were schools that were similar to Success 
schools in many metrics (such as the 5Essentials and middle grades on-track rates) the year prior to 
implementation. 


e Finally, the third group consists of all other elementary schools part of the same networks as the 
Success Project schools. This group of network schools consists of 129 elementary schools which are in 
Networks 3, 5, 8, 9, 10, and 11. Since these schools are in the same networks as Success schools, they 
were subject to similar guidelines from the network office as the Success schools. Fourteen of the 
invited schools were in these networks; they are included as part of the invited schools not as part of 
schools in the same networks. 


Reading the graphs: The majority of the figures included in this chapter are a series of bar graphs. Four pairs of 
bars are depicted in the graphs. Each of the four pairs represents a group of schools. The first pair represents the 
Success schools, the second pair represents the groups of AUSL schools, the third pair represents the invited 
schools, and the last pair represents other schools in the same networks as the Success schools. Each bar 
represents the increase or decrease in the outcome in a particular year compared to the trend in the outcome 
observed in the years prior to implementation. The first bar in a pair corresponds to the changes in the first 
(partial) year of implementation, 2014-15, and the second bar in the pair represents the changes in the second 
year of implementation, 2015-16. The outcomes averages are shown in Appendix A. On each bar, there are 
whiskers representing the standard error around that bar’s estimated value; the bottom whisker is the lowest 
expected value for the bar, and the top whisker is the highest expected value for the bar. On the GPA changes 
graphs, for example, expected change in students’ GPAs can fall within the range of the whiskers and could be as 
low as the bottom whisker or as high as the top whisker. 


More details on the data, methods, and statistical models used for this study are provided in Appendix A and 
Appendix B. 


A_ Three of these schools became AUSL schools, and therefore turnaround, in the 2014-2015 school year. 
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We identified students who were in sixth, seventh, or eighth grade during an academic year to be the focus of the 
analyses. If students transferred schools during the year, we assigned them to the school where they enrolled for the 
most number of days during that academic year. In the year before the intervention (2013-14), students in the 
analytic sample were in schools for 169 days, on average, in the Success schools. The students in the other groups 
of schools were enrolled for a similar number of days, on average. Students from other invited schools were enrolled 
for 170 days, on average, for 171 days for other schools in the same networks, and for 167 days for students in 
AUSL schools (see Appendix A for more details). The analyses of the survey data restricted the sample to students 
who reported about the school where we assigned them, which is a large percentage of students given how many 


days, on average, they were attending these schools. 


Success schools experienced higher attendance rates in Year 1 and Year 2 than in prior years; other 
schools followed similar patterns. Figure 2 shows the estimates for changes in attendance rates compared to 
the rates in prior years. Attendance rates in Success schools increased by 1.2 percent in Year land by 2.1 percent in 
Year 2, above the growth that these schools were experiencing in the three years before the program. These estimates 


for the Success schools were statistically different from zero. 


We see a similar pattern for the other groups of schools consistent with CPS’s effort to improve attendance across 
the district during the last few years. The increases in attendance rates were higher for Year 2 than for Year 1, most 
different from zero. Even though the estimates from the Success schools were larger than the estimates for any other 


group, statistically these attendance rates estimates were no different than the estimates for other schools. 


FIGURE 2 
Changes in Attendance Rates in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences. See Appendix B for more details. 


There were no improvements in reading, math, or core GPA in the Success schools; other schools also 
did not see improvements in students’ GPAs. Figures 3, 4, and 5 show the values for these three outcomes. 
Changes to reading and math GPAs were small in Year 1 with slightly higher increases in Year 2 for reading and 
math. But these increases were not statistically significant. Core GPA remained similar to prior years. Though one 
of the program’s goals was to help school staff and students understand the importance of grades and improve 


students’ GPAs, Chapter 2 highlights that coordinators experienced several challenges in this aspect of their work. 


JCHICAGO Consortium Research Report | The Success Project 


31 


$2 


AUSL schools that implemented some of the elements of the Success Project saw decreases to their GPAs in both 
years although not significantly different from zero in Year 2. Changes to GPA for the invited schools and schools 


within the same networks were small and non-significant in most cases. 


FIGURE 3 
Changes in Reading GPA in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed differences with respect to AUSL schools in both 
years (p-value=0.025 for 2014-15 and p-value=0.023 for 2015-16). See Appendix B for more details. 


FIGURE 4 
Changes in Math GPA in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences. See Appendix B for more details. 
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FIGURE 5 
Changes in Core GPA in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences. See Appendix B for more details. 


Middle grades on-track rates went up both years, especially in Year 2, in the Success schools; smaller 
increases were evident in other groups as well. As with other outcomes shown, the Year 2 increases were 
larger than the first year for all schools (see Figure 6). Most of those increases were all significantly different from 
zero (except for AULS schools in Year 1). Even though the Success schools saw the largest increases among these 
groups, given the variability in the estimates, the statistical tests do not show that their increases were different from 


the invited schools or schools in the same networks. 


FIGURE 6 
Changes in Middle Grades On-Track Rates GPA in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed differences with respect to AUSL schools in both 
years (p-value=0.049 for 2014-15 and p-value=0.080 for 2015-16). See Appendix B for more details. 
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In Year 2, students in the Success schools were more likely to report exhibiting academic behaviors than 
previously; students in other schools did not report higher levels of these measures. Students in Success 
schools reported that they set aside time for schoolwork and give priority to studying, as well as persevering in their 
work at higher rates than students in these schools prior to the intervention. Through the 6to16 curriculum and the 
focus on grades, coordinators might have influenced how much work and perseverance students were willing to put 
toward their school work. Even though we did not find much improvement in students’ GPAs, this improvement 
in students’ reports of working hard and persevering could indicate that the Success Project may have influenced 
students’ academic behaviors. In fact, one of the lessons in the 6to16 curriculum taught students how to set short- 


term and long-term goals and create action plans to accomplish those goals. 
For students in other comparison schools, there were no changes to their reports in these two measures (see Figure 


7). In all cases the differences with prior years were statistically not different from zero. 


FIGURE 7 
Changes in Reports of Study Habits and Grit in Year 1 and Year 2, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed differences in 2015-16 with respect to the Invited 
schools and schools within the same networks (Study habits: p-value=0.021 with respect to the Invited schools and p-value=0.090 for the same network 
schools. Grit: p-value=0.063 with respect to AUSL schools). See Appendix B for more details. 


Students in the Success schools reported improvements, especially in Year 2, in school climate toward 
a supportive environment compared to years prior to the intervention; in most cases students in other 
schools reported similar levels of these measures compared to prior years. Students’ perceptions about their 
peers, teachers, and school community improved in the Success schools (see Figure 8). Students’ answers to the 
surveys indicated that their peers demonstrated behaviors that lead to academic achievement more than in the years 
before the intervention, that teachers connected with students in the classroom and supported them in achieving 
academic goals, that students and teachers shared a higher level of mutual trust and respect, and students felt more 


included in their school’s community than in the years prior to the intervention. 
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These positive changes in school climate were not observed in most of the other schools in the comparison groups, 


suggesting that some of the changes were not being driven by trends in the district but, perhaps, in changes that 


took place in the Success schools. The changes in the Success schools were statistically different from the changes 


in some of the other schools. In particular, the changes in the measure of school connectedness in Year 2 was 


statistically different and higher than in all other three groups of schools. 


FIGURE 8 


Changes in Reports of Peer Support for Academic Work, Academic Personalism, Student-Teacher Trust, 
and School Connectedness in Year 1 and Year 2, Compared to Prior Years 
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Panel C: Student-Teacher Trust 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and fifth-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed differences in 2015-16 (Academic Personalism: p- 
value=0.091 with respect to Invited schools. Student-teacher trust: p-value=O.069 with respect to schools in the same network. School connectedness: p- 
value=0.015 with respect to AULS schools, p-value=0.025 with respect to Invited schools and p-value=0.006 with respect to schools in the same networks). 


See Appendix B for more details. 


In sum, most of the indicators point to positive changes in the Success schools, especially in Year 2. Coordinators 


did not start in their schools until the second semester in Year 1, which, along with other implementation challenges 


discussed in the previous chapter, could have limited the impact of the program in the first year. In Year 2, greater 


impact could be the result of 1) the coordinators being in their schools from the start of the school year and/or 2) 


seventh- and eighth-grade students having some familiarity with the program from the prior year. 


Research Report | The Success Project 


$5 


S56 


Students’ reports of their own priorities for schoolwork and how they perceived their peers’ behavior toward 
academic achievement improved in Success schools, compared to prior years. Students also reported increased 
connection with teachers and increased teacher support in their academic achievement. In addition, Success schools’ 
students reported that they felt more connected to their school community. These are some positive signs that the 


culture in these schools was changing to become more conducive to student success. 


Most of the student-level outcomes that the program targeted showed signs of improvement. Given the small, 10- 
school sample and the newness of the program, it is difficult to measure whether these changes were larger than 
what other schools experienced without this kind of intervention. Most interventions need three to five years to 
fully see the effects on students’ outcomes.?' However, these changes point to movements in these schools in the 
right direction. One area were changes were not so evident was students’ GPAs. Notably, coordinators reported in 


both years that this was the lever where they struggled the most. 


High School Attended by Eighth-Grade Graduates 


This section focuses on exploring the high school selection of eighth-grade graduates for three cohorts before the 
intervention and the two cohorts that were in the Success schools in Year 1 and Year 2. For this analysis, we focus 
on students who were in eighth grade and attended a CPS high school the following fall. We restrict the sample to 
those students who were in eighth grade in the same school in the fall and the spring, since the high school selection 
process starts early in the academic year with applications being due in early December and decisions taking place 


in the spring. Appendix A shows the number of eighth-grade graduates that are part of the analytic sample. 


One lever in the Success Project’s theory of action was for students to engage in the process of high school choice 
and to increase the percent of eighth-graders who applied to selective enrollment and other “good match and fit” 
high school options. This focus may have increased the number of high schools attended by schools’ eighth-grade 
graduates, so we reviewed high school enrollment data before and after the program was implemented and offer the 
number of high schools for the other groups of schools. Table 2 shows those numbers and also the variability in the 


range of number of high schools attended (numbers in parenthesis). 


While it is hard to compare the numbers across groups, one can compare the numbers over time in one group in 
contrast with changes in the other groups. The number of high schools attended depends on the number of eighth- 
grade graduates. In addition, the neighborhood where elementary schools are located or where students lived could 
have also influenced the number of high schools attended by graduates. In some areas of the city it is easier to access 
a larger number of high schools, due to accessibility to public transportation, the number of high schools nearby, 
and/or the opening of new schools in those neighborhoods. In more isolated areas of the city the number of high 


schools attended could be limited. 


On average, the 2012 eighth-grade graduates from a Success school attended 15 different CPS high schools. The 
number of high schools that students in Success schools attended increased in 2014 to 22 different CPS high schools, 
the year just before the implementation of the Success Project. This increase is higher than all comparison groups, 
some of which remained flat. This increase may have been the result of an increase in the number of eighth-grade 
graduates in these schools that year, since half of Success schools became welcoming schools and therefore received 
an influx of students from the 47 elementary schools that were closed at the end of the 2012-13 academic year. The 


number of graduates increased by almost one-quarter. 


21 Aladjem et al. (2010); et al. (2006); Borman, Hewes, Overman, & Brown (2003); Desimone (2000); Zhang, Fashola, Shkolnik, & Boyle 
(2006). 
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For the two cohorts that experienced the Success Project, the numbers of high schools attended increased slightly, 
compared to the cohorts joining high schools in the fall of 2012 and 2013 and compared to the other groups of 
schools where the number of high schools attended has been very steady over time. Graduates at the other schools 
attended a similar number of schools over time, not that different from the Success schools, although they did not 


experience an increase in student population and an increase in the number of high schools attended. 


TABLE 2 
Number of CPS High Schools Attended by Eighth-Grade Graduates 


Cohorts Prior to Implementation of Cohorts During Implementation of the 
Success Project Success Project 


Fall of 2012 | Fall of 2013 | Fall of 2014 Fall of 2015 Fall of 2016 
15 16 IP 18 19 


Success Schools 


(8 to 23) (10 to 25) (13 to 32) (12 to 27) (12 to 25) 
18 19 19 19 17 

AUSE Schools (9 to 31) (9 to 31) (11 to 29) (10 to 28) (6 to 30) 
F 16 16 16 16 iS 

Invited Schools (6 to 24) (6 to 26) (6 to 25) (6 to 29) (5 to 24) 
Schools in Same 17 17 18 17 17 


Network (2 to 38) 


(4 to 38) (4 to 37) (3 to 43) (2 to 36) 


Note: The number on top in each cell shows the average number of different CPS high schools attended by eighth-grade graduates. The numbers in 
parentheses show the range: the fewest CPS high schools and the most CPS high schools attended by eighth-grade graduates in that school group. 


Besides looking into the number of high schools in which these students enrolled, it is important to understand the 
kinds of high schools they attended. Figure 9 shows the distribution of students by the different types of high 
schools. Each bar represents a particular cohort and each group of bars represents a group of schools: Success 
schools, AUSL schools, invited schools, and schools in the same network. The types of high schools are divided by 
whether students attended their own attendance area high school, other neighborhood high schools, career 
academies, charter high schools, other high schools with no attendance area boundaries, and selective enrollment 


high schools. 
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FIGURE 9 
Percent of Students Attending Different Types of High Schools 
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Note: Years refer to the fall of the ninth-grade year when eighth grade graduates joined a high school. For example, 2012 refers to the fall of 2012 when the 
eighth-grade graduates of spring 2012 joined a high school for their ninth-grade year. 


There are four notable findings regarding types of high schools attended by Success schools’ students, as captured 


in Figure 9. 


In the fall of 2015, eighth-grade graduates from the Success schools were more likely to attend a charter 
high school (32 percent) than in prior years (between 18 and 22 percent). The Year 2 eighth-grade graduates 
were still more likely to attend charter high schools than prior to the intervention, although that number was smaller 
than for the Year 1 eighth-grade graduates (26 percent). This pattern was similar for eighth-grade graduates from 
AUSL schools and the invited schools, but not for other schools in the same network. As previously noted, the 
opening of a new charter school nearby might drive increases in the percent of students attending the new school. 
During this period, the district opened many charter schools and this could explain some of these increases.2* 
Another explanation for the increase in the Success schools is that at the time the coordinators joined schools in 
Year 1, students had already applied to schools and most probably openings in high schools toward the end of the 


year were in charter schools and other neighborhood schools still with open seats. 


The increase in enrollment in charter schools came with a decrease in enrollment in other neighborhood schools 
and career academies for the Success schools graduates. Students in the Success schools were already becoming less 
likely to enroll in their assigned neighborhood high school; this started in 2014. Enrollment in students’ own 
attendance area high school remained at the similar level as in 2015 when the schools became Success schools. For 
graduates of A USL schools and the invited schools, the increase in attendance in charter high schools came with a 
decrease in enrollment in other neighborhood schools and career academies, as well as a decline in enrollment in 
students’ own neighborhood schools. Very few of the AUSL graduates attended their own attendance area high 


school. 


22 Gwynne & Moore (forthcoming); Barrow & Sartain (forthcoming). 
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The percent of Success school graduates who enrolled in selective enrollment high schools increased 
slightly in the years of the intervention; we found similar patterns in other schools. For the Year 1 and 
Year 2 eighth-grade graduates, there was an increase in the number of students enrolling in selective enrollment 
high schools. This kind of growth was happening even before the intervention, a 2 percent increase yearly. Similar 
increase took place in 2015 and 2016 in the other schools. Even after controlling for the characteristics of the eighth- 
grade graduates in these schools, the changes after the intervention in the Success schools were no different from 


other schools (see Figure 10). 


FIGURE 10 


Changes in Enrollment Rates in Selective Enrollment High Schools for Year 1 and Year 2 Eighth-Grade 
Graduates, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and seventh-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences. See Appendix B for more details. 


While the information on the type of high schools students enrolled gives us an initial picture of the high school 
selections and how they have changed, it does not tell us much about the particular qualities of those choices. Next, 
we explore some of the characteristics of the high schools in terms of ACT and graduation rates. These metrics are 
not perfect metrics for education quality, but are meaningful and have been found to be predictive of how students 
will do in high school.23 We will use the information available to students about the high schools when they were 


applying to and selecting high schools.24 


23 Allensworth, Moore, Sartain, & de la Torre (2016). 
24 For example the cohort of eighth-grade graduates joining high school in the fall of 2014, applied to high school during the 2013-14 
academic year and the information available to them was from the 2012-13 academic year. 
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The high schools attended by Success schools’ eighth-grade graduates had similar average ACT scores 
than those attended by graduates from prior cohorts. In contrast, eighth-grade graduates from the same 
network schools attended schools with lower ACT scores, compared to prior cohorts. Figure 11 shows the estimates 
for all groups. While the changes were positive for the 2015 cohort from the Success schools, the values were not 
different from zero for either cohort. AUSL graduates attended high schools with higher ACT scores than prior 
graduates, but the graduates from the invited schools and schools within the same network attended high schools 
with lower ACT scores and in the case of the graduates from the same network schools, those values were 


statistically different from zero. 


FIGURE 11 
Changes in High School Average ACT of High Schools Attended by Year 1 and Year 2 Eighth-Grade 
Graduates, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and seventh-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences, except for the 2014-15 cohort, with 
respect to the same network schools (p-value=0.017). See Appendix B for more details. 


Eighth-grade graduates from the Success schools attended high schools with lower graduation rates, 
compared to trends from prior years. Figure 12 shows the changes in high school graduation rates of high 
schools attended by eighth-grade graduates. The same trends were observed for other groups of schools as for the 
Success schools; the graduation rates of high schools attended were lower than the trends observed for cohorts prior 
to the 2014 cohort. The trends observed for the graduates from Success schools were no different than what was 


observed for graduates from other schools. 


Chapter 3 | Middle Grades Outcomes and High School Selection in the First Two Years 


FIGURE 12 


Changes in High School Graduation Rates of High Schools Attended by Year 1 and Year 2 Eighth-Grade 
Graduates, Compared to Prior Years 
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Note: Estimates come from a statistical model where we controlled for students’ gender, race, whether students were receiving special education services, 
whether they were old-for-grade, socioeconomic status, and seventh-grade math and reading scores. The whiskers on each bar represent the standard error 
around that bar’s estimated value; the bottom whisker is the lowest expected value for the bar, and the top whisker is the highest expected value for the bar. 
Statistical tests for whether the changes in the Success schools were different from other schools showed no differences. See Appendix B for more details. 


In sum, the 2015 and 2016 cohorts of eighth-grade graduates attended charter schools at higher rates than in the 
past, and fewer students attended other neighborhood schools and career academies as a result. The high schools 
attended by Success schools’ students had similar ACT scores, but lower graduation rates than the high schools 
attended by prior cohorts of Success schools’ graduates. These changes were no different than what was observed 


for graduates of the other groups of schools we included in our analyses. 


The most relevant information for understanding the initial effectiveness of the program beyond the middle grades 
would be to know how Success schools’ students performed while in high school. One of the program’s goals was 
to find a “good match and fit” high school for each student that would help students have a good transition to high 
school and put them on a path to be college ready. However, as of the writing of this report, only one cohort of 
graduates have attended high school for a full year; the second cohort is currently in their ninth-grade year. Their 
high school performance will be a good measure of whether students found a “good match and fit” high school. 


Unfortunately, it is too early to study these outcomes. 
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CHAPTER 4 


Interpretive Summary 


In Chicago and across the country, educators and policymakers are 
giving increasing attention to the importance of grades and attendance 
for students at all grade levels. Efforts to equip all students to be 
college and career ready continue to gain traction. Concurrently, school 
choice remains a highly-debated lever for school reform and school 


improvement. The Success Project’s experiences working on grades, 
attendance, and high school choice with middle grades students in 
neighborhood schools offer valuable insights for other programs that 
also seek to address these areas with students. 


Implications for Introducing a Curriculum Aimed at Improving High School and 
College Knowledge in Schools 

Knowing the school context is important for successful implementation. Any new initiative will likely be 
implemented alongside other programs and school priorities. This can make it difficult to find and protect time for 
the new initiative and to implement a program as originally intended. The Success Project staff and coordinators 
learned that in order for the 6to16 curriculum to be implemented in all 10 of its schools, the curriculum needed to 
be flexible. It had to be able to be adapted for schools, because each had a unique set of circumstances and 
constraints. The flexibility included the length of the class period and whether or not the curriculum would be 
delivered in a whole classroom setting or to small groups of students. Success coordinators felt most successful 


when they had a fully-developed curriculum that they could modify to meet the needs of their schools. 


The course content needs to be relevant, accessible, and engaging for students. Students in different 
schools enter programs with diverse experiences and different levels of knowledge on particular subjects. Success 
Project staff learned in Year 1 of implementation that the 6to16 curriculum assumed too much college knowledge 
from their students and therefore did not provide sufficient scaffolding for them to engage in many lessons. This 
created challenges for coordinators; it made it difficult for them to engage their students in their 6to16 classes and 
it made it difficult for students to learn the 6tol16 content. Coordinators felt most successful when their students 
could fully engage with the content. Programs could consider testing their curriculum, or parts of their curriculum, 


in the contexts in which the curriculum will be delivered, to allow for improvements before full implementation. 
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Even a supplemental curriculum requires teaching and classroom management skills. The 6to16 curriculum 
was largely delivered by the Success Project coordinators. These coordinators joined the Success Project with a 
variety of backgrounds; some reported that they sought out, and benefited from, instructional coaching and 
professional development. Classroom management was the biggest self-reported instructional issue, and 
coordinators felt that their classroom management struggles hindered student engagement and undermined buy-in 
for the Success Project in their schools. They also reported that addressing these challenges resulted in much greater 
successes. Other programs could benefit from considering the kinds of supports that their staff may need to 


successfully deliver a similar curriculum. 


Implications for Programs Working on Improving Grades and Attendance 

Students can become empowered to understand the importance of their grades and attendance, and to 
track their own data. Some of coordinators’ biggest self-reported successes were around working with students to 
improve their grades and attendance. Once students understood how their GPA, attendance rates, and, 
consequently, their middle grades on-track status, were calculated, students felt empowered to take actions that 
would improve their own grades (e.g., turn in missing assignments) and advocate for themselves with their teachers. 
Often, teachers are in charge of monitoring students’ grades and attendance, and are the ones supporting and 
encouraging students to complete assigned work; this is a lot to do. Having an additional staff member equip 
students with the knowledge and skills they need to monitor their own academic progress could be helpful to both 


students and teachers. 


Structures that engage and support teachers may help them change practice. While coordinators reported 
seeing improvements in students’ ownership of their grades and their academic behaviors, working to improve 
students’ grades was one of the hardest pieces of the project; coordinators reported struggling to engage teachers in 
these efforts. Teachers may not have supports, nor opportunities, to know how to work with students to improve 
their grades and attendance. For example, the role of the Success coordinator with respect to this task varied by 
school; in some they were able to meet with teams of teachers and share data, in some other schools they were more 


limited in their engagement of teachers. 


Teachers may benefit from additional information about the importance and implications of working to improve 
students’ grades. Many teachers in Success schools had not previously engaged in conversations with students about 
why they were absent from class, why they did not turn in an assignment, or why they were not mastering material. 
Thus, teachers may need some support to have these kinds of conversations with their students. For some, having 
these conversations with students offered them an opportunity to build different kinds of relationships with students. 
In addition, coordinators learned that there had to be trust between the teacher and the student in order for them to 
have an honest and sometimes vulnerable conversation with one another. Coordinators found scripts to be useful 
for these conversations; scripts helped the student and teacher focus on the problem and solutions without either 


party feeling attacked or disrespected. 


It takes a school-wide effort to work on improving students’ grades. Coordinators learned that they alone 
could not create rapid improvements in grades and attendance. They needed their administrators, teachers, and 
students to buy into the importance of grades, and engage in improvement efforts. Structures that facilitated this 
buy-in—including time set aside for teachers and staff to talk about students’ grades and attendance and who could 
intervene—were critical to getting more teachers and staff involved in the work. In the absence of a coordinator, it 
could be quite challenging for school staff to meet with every student individually about their grades and attendance, 
but establishing ways to group students and match them with interventions may effectively allow staff to reach and 


support every student. 
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While improving student attendance was directly connected to Success schools’ school-level goals, such as 
improving or maintaining accountability ratings, or getting students more engaged in their classwork, improving 
students’ grades was not as directly tied to schools’ accountability ratings or their top priorities. Some coordinators 
found ways to capitalize on the existing momentum and systems already in place in their schools, but creating 
systems and structures to support grades interventions was difficult work, which may explain why we did not see 
improvements in core GPA, even though we saw signs of small improvements in most other middle grades 


outcomes. 


Implications for Programs Working on High School Application and Selection 

Fully supporting students in the high school application and selection process is time-intensive. CPS 
moved to a common application for high schools starting in the 2017-18 school year, which should reduce some of 
the burden that coordinators experienced, including faxing students’ records to multiple places, securing parents’ 
signatures on multiple applications, and helping students fill out multiple applications. Even with this change, the 
high school application and selection process is time-consuming. Before students even begin to fill-out an 
application, they have to research what their options are and decide where they want to apply, and then the 
application needs to be filled-out. When students learn what high school or high schools they have been accepted 
to, they have to select the high school they will attend. Coordinators were most successful when they had systems 
in place for each stage of the application and selection process and when they established clear roles with the 
counselor in their schools, as well as with the students and their families for the level of coordinator involvement 
that they expected and needed in the process. Even when those systems were in place, coordinators reported 


spending a lot of time helping their eighth-grade students. 


Programs would benefit from clear planning for family engagement. Families are the ultimate decision- 
makers about where students attend high school. This means that schools and programs need to intentionally engage 
families if they want to fully support students in the high school application and selection process. Coordinators 
found that some families did not have much or, in some cases, any information on the high school application 
process or on schools that were options for their children. Some coordinators also reported that parents had personal 
connections to high schools; parents also know their children and have considerations of what schools would be a 
“good match and fit” for their child—often considerations beyond what the program was considering. In addition, 
some coordinators were more successful than others with family involvement in this process; it required 
coordinators to invest time with families to hear their preferences, concerns, and questions, and also sometimes 
meant that coordinators were providing families with current school information that contradicted their prior 
knowledge. Coordinators were most successful when they worked with families to think about where a student 
would be most successful. Other programs that want to influence high school application and selection would benefit 


from considering when and how to involve and partner with parents in the application and selection process. 


The best high school for a student is not always clear. Coordinators reported that some of their biggest 
successes had been around high school placements. In its inception, and even in its recruitment, the program aimed 
to increase the number of eligible students applying to and enrolling in selective enrollment high schools. Behind 
this push was the idea that these schools were going to be the best match and fit for those students. The program 
staff learned quickly that selective enrollment high schools were not realistic options for many of their students. 
Even though many of their students were invited to sit for the admissions exam, it was highly unlikely that many 
of those students would earn enough points on the admissions exam to qualify for enrollment at the selective 


enrollment high schools they were considering. 
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Additionally, for some students who were truly eligible, coordinators learned that selective enrollment high schools 
were not always the best fit for these students. The Success Project came to understand that while thinking about 
course offerings, academic and extracurricular programs, and school quality (e.g., graduation and Freshman 
OnTrack rates) were part of the definition of a “good fit” high school, the program needed to broaden their definition 
of “fit” to consider families’ preferences and constraints, as well as the student’s commute and safety. It is unrealistic 
to think that every school in the city of Chicago is an option for every student. Coordinators learned that they could 
not decide what a student’s best option would be. High school selection was a unique decision for every student 


and every family. 


General Implications for School-Based Programs 

Buy-in from school leaders facilitates new initiatives in schools. In Year 1 and Year 2, coordinators reported 
that when the principal was on board with the ideas and goals of the Success Project, they were better able to fulfill 
their expected responsibilities and meet their expected goals. This finding is consistent with the literature on the 
implementation of education programs and reforms. Despite its importance, many coordinators reported that 
establishing their administrator’s buy-in was a challenge in many of their schools. By initially inviting schools to 
apply to participate in the program, the program staff thought it would ensure some level of school and administrator 
buy-in from the get-go; they believed that only schools and administrators who believed in the program’s goals and 
only schools that could commit the needed resources would apply. However, even when there was initial buy-in by 
school leaders, principals have demanding jobs and deal with many different issues every day. These competing 
demands and pressures sometimes lead to the Success Project not taking priority. Additionally, some schools 
experienced principal turnover, which required the coordinator and project to re-introduce their work and goals to 
the administration and begin the investment work again. Without principal and staff buy-in, there is a risk that 
programs like the Success Project will only be partially implemented and/or may not be prioritized within the school 


the way the program hopes. 


Achieving impact takes time. As with any other initiative, the Success Project took time to fully implement. It 
took time to secure buy-in and to bring everyone in the Success schools on board with the goals of the project. In 
fact, the literature on program implementation suggests that many initiatives take three to five years to take hold in 
schools. In their second year, coordinators reported that some of their administrators, teachers, and students’ 
attitudes and beliefs started to change as they saw some initial results. In other words, as they started to see how 
things could change, they started being supportive of the project and its goals. Our quantitative analyses show that 
there were small and positive, though statistically insignificant, initial results, including improvements in school 
climate (e.g., student-teacher trust and peer support), student self-reports of their study habits and grit, and student 
attendance rates. These improvements may have led to improvements in other outcomes (e.g., core GPA) in future 
years if the program would have continued. It takes time to build a foundational level of trust and buy-in and to 
adapt a program to meet the needs of particular contexts. Consequently, while the Success Project may have been 
building this kind of foundation, we would expect that it would take time before we would see significant positive 


results. 


We hope that this report provides some insight into the implementation of the Success Project. As other 
organizations and programs attempt to build students’ college knowledge and cultural capital, improve students’ 
grades and attendance, and utilize high school choice to improve student outcomes, we hope they can learn from 


the implementation, experiences, and initial evidence of the Success Project as they design their interventions. 
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Appendix A 


Quantitative Data and Sample 


This appendix contains information on the data sources and variables used for the quantitative analyses in this 


report as well as how the sample of students was selected. 


Quantitative Data 


Student data came from CPS from 2011-12 to 2015-16 administrative data, test data, transcript data, and survey 


data. 


Student addresses were linked to data from the 2010 U.S. Census Bureau at the block-group level. The 


concentration of poverty in a student’s neighborhood and social capital were calculated using 2010 U.S. Census 


Bureau figures. School-level data comes from publically-available data downloaded from the CPS website. Student 


variables and outcomes are defined as follows: 


School enrollment. School student attended. 

Race. Whether a student was Asian, Black, Latino, White, or other. 

Gender. Whether a student was male or female. 

Grade. Student’s grade. 

Special education. Whether a student was receiving special education services. 


Old-for-grade. A dummy variable to indicate whether a student was older than would be expected for her 


grade, based on school system guidelines and the student’s birth date. 


Neighborhood concentration of poverty. Based on data from the 2010 U.S. Census on the census block 
group in which students lived. Students’ home addresses were used to link each student to a particular 
block group within the city, which could then be linked to census data on the economic conditions of the 
student’s neighborhood. Two indicators were used to construct these variables, the log of the percentage 


of households above the poverty line and the log of the percentage of men employed in the block group. 


Student prior performance. For the analyses of middle grade outcomes, this was the students’ fifth-grade 
reading and math test scores based on the Illinois Standards Achievement Test (ISAT) score and the 
NWEA test scores when ISAT data was not available. ISAT was given until the spring of 2014. For high 
school choice analyses, this variable was the student’s seventh-grade score, since this score is used for high 
school applications. To make scores comparable across tests, scores from either the ISAT or NWEA were 


converted into standard deviations from the system mean in each year and for each grade. 


Attendance rates. Number of days a student was present, divided by the number of days the student was 


enrolled in school, multiplied by 100. 


Reading, math and core GPA. GPA was calculated on a four-point scale (where A=4, B=3, etc.). Core 


GPA included grades from the following subjects: Reading, math, science, and social studies. 


Middle grades on-track. This is a metric used in CPS for students in grades 3-8. We use the same definition 
as CPS, but we call it middle grades on-track because we only apply it to students in grades 6, 7, and 8. 
Students are on-track in grades 3-8 when their attendance is 95 percent and their reading and math grades 


are C or better. We only include students with at least a grade in reading or math. 
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e Survey measures. These include study habits, grit, peer support for academic work, academic personalism, 


teacher-student trust, and school connectedness. See Table A.1 for details on the items that comprise the 


measures and their reliabilities. 


e Types of high schools. Whether a high school was a neighborhood high school, career academy, charter 


high school, selective enrollment, or other type. If a student was attending a neighborhood high school, we 


checked to see if it was his or her attendance area high school. 


e School average ACT. The published average ACT composite score at the school level. 


e School graduation rates. The published four-year graduation rate of high schools. 


TABLE A.1 
Survey Measure Details 


How much do you agree with... 


Study Habits 
Reliability = 0.76 


Grit 
Reliability = 0.72 


Peer Support for 
Academic Work 
Reliability = 0.73 


Academic Personalism 
Reliability = 0.70 


Teacher-Student Trust 
Reliability = 0.63 


School Connectedness 
Reliability = 0.78 


| set aside time to do my homework and study 

I try to do well on my schoolwork even when it isn’t interesting to me 
If | need to study, | don’t go out with friends 

| always study for tests 


(Response categories: Strongly Disagree, Disagree, Agree, Strongly Agree) 


To what extent do the following describe you... 


| finish whatever | begin 

lam a hard worker 

| continue steadily toward my goals 
| don’t give up easily 


(Response categories: Not like me at all, Not much like me, Somewhat like me, 
Mostly like me, Very much like me) 


How many students in your [target] class... 


Feel it is important to come to school everyday 
Feel it is important to pay attention in class 
Think doing homework is important 

Try hard to get good grades 


(Response categories: None, A few, Some, About half, Most, All) 


The teacher for your [target] class... 


Helps me catch up if | am behind 

Is willing to give extra help on schoolwork if | need it 

Notices if | have trouble learning something 

Gives me specific suggestions about how | can improve my work in this class 


(Response categories: Strongly Disagree, Disagree, Agree, Strongly Agree) 


How much do you agree with... 


My teachers always keep his/her promises 

| feel safe and comfortable with my teachers at this school 

When my teachers tell me not do something, | know he/she has a good reason 
My teaches will always listen to students’ ideas 

My teachers treat me with respect 


(Response categories: Strongly Disagree, Disagree, Agree, Strongly Agree) 


How much do you agree with the following statements about your school... 


| feel like a real part of my school 

People here notice when I’m good at something 

Other students in my school take my opinions seriously 
People at this school are friendly to me 

I’m included in lots of activities at school 


(Response categories: Strongly Disagree, Disagree, Agree, Strongly Agree) 


Note: Reliability statistics are reported for the survey data collected in the spring of 2015. 
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Analytic Sample 

We identified students who were in sixth, seventh, or eighth grade during an academic year to be the focus of the 
analyses for middle grade outcomes. If students transferred schools during the year, we assigned them to the school 
where they enrolled for the most number of days during that academic year. Table A.2 shows the number of students 


included in the analyses and Table A.3 contains the number of days they were enrolled in school. 


For the analysis of high school choices, we focused on students who were in eighth grade and attended a CPS high 
school the following fall. We also restricted the sample to those students who were in eighth grade in the same 
school in the fall and the spring, since the high school selection process starts early in the academic year with 
applications being due in early December and decisions taking place in the spring. Table A.2 also contains the 


number of graduates included in these analyses. 


TABLE A.2 
Number of Students Included in the Analyses 


Success Schools (10 Schools) 
- Number of middle grade students 1,690 1,635 1,959 1,841 1,774 


- Number of eighth-grade graduates 431 449 558: 496 476 
AUSL Schools (23 Schools) 

- Number of middle grade students 3,718 SISO S27 3,811 3,639 

- Number of eighth-grade graduates 988 1,076 1,061 1098 882 
Invited Schools (28 Schools) 

- Number of middle grade students 507s 4,787 5,153 5,056 4,922 

- Number of eighth-grade graduates 1,478 1,362 1,387 1422 1,309 


Schools in the Same Network (129 Schools) 


- Number of middle grade students 24,50 24,396 25259) 24,498 23,850 
- Number of eighth-grade graduates 6,520 6,658 LAN 6,896 6,320 
TABLE A.3 


Average Number of Days Enrolled in School for Middle Grades Students in the Analytic Sample 
(standard deviation in parentheses) 


162 173 169 171 170 

Success Schools (10 Schools) (23) (25) (27) (27) (23) 
160 171 167 169 165 

. 163 174 170 173 168 
Invited Schools (28 Schools) (24) (26) (26) (26) (28) 
Ae ares rs as oan 164 175 171 173 170 
Schools in the Same Network (129 Schools) (22) (23) (25) (24) (22) 
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Appendix B 


Methods 


To examine the initial effectiveness of the Success Project, we ran analyses that contrasted students’ outcomes 
before and after the initiative took place, and compared these changes with changes in different groups of schools. 
We used models that incorporated school fixed effects. This analysis model allows schools to be compared to their 
own prior outcomes. The comparison schools provided a further contrast—kind of a difference-in-difference 
approach—so that readers of this report can gauge whether these changes were similar to what other schools 


experienced in those years or whether the changes were higher or lower compared to other schools. 


We offer three other groups of schools for comparison. The first group consists of all the AUSL elementary schools. 
These schools implemented a model with some elements similar to the Success Project. In the 2014-15 year, 23 
AUSL schools were part of this group and those are the schools included in the analyses. Four more AUSL schools 
implemented this program in 2015-16, but they are not included in the analyses because they started a year later 


and three of them became turnaround schools the year prior. 


The second comparison group includes the group of schools that were invited to apply to the Success Project. The 
comparison group is formed by 28 schools. These were schools that were similar to Success schools in many metrics 


(such as the 5Essentials and middle grades on-track rates) the year prior to implementation. 


Finally, the third group consists of all other elementary schools part of the same networks as the Success Project 
schools. This group of network schools consists of 129 elementary schools which are in Networks 3, 5, 8, 9, 10, 
and, 11. Since these schools are in the same networks as Success schools, they were subject to similar guidelines 
from the network office as the Success schools. Fourteen of the invited schools were in these networks; they are 


included as part of the invited schools not as part of schools in the same networks. 


The models also include student-level covariates to adjust for any changes in the types of students who were 
attending the schools over the period being analyzed. Thus, the model shows the average outcomes adjusted for 
changes in the types of students in the school. The student covariates included in the model are a dummy variable 
for students’ gender, a series of dummy variables for students’ race, whether students were classified as receiving 
special education services, whether they were old-for-grade, students’ neighborhood concentration of poverty, 
students’ neighborhood social status, and prior reading and math scores. For middle grade outcomes, the prior 
reading and math scores were those of the students in fifth grade, just before entering the middle grades. For high 
school choice analyses the prior reading and math scores were those in the seventh grade, the data available when 


students are making decisions on where to apply and attend high school. 
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The model is described as follows: 


Outcome; = ,; + ai*(Success Schools*trend jj) + a 2*( Success Schools* Year 2014-15,-) + a3*( Success Schools* Year 
2015-16; ,) + 


Bi*(AUSL Schools*trend,;) + B 2*( AUSL Schools* Year 2014-15) + B 3*( AUSL Schools* Year 2015-16;;) + 
yi*(Invited Schools*trend,-) + y 2*(Invited Schools* Year 2014-15;) + y 3*(invited Schools* Year 2015-16) + 


ui*(Same Network Schools*trend,;) + \2*( Same Network Schools*Year 2014-15;:) + ps*( Same Network 
Schools* Year 2015-16;;) + 


mi*(malej,) + m2*(white;,) + m3*(Asiany:) + ma*(Latino,y,) + m5*(special education;;) + mo*(old for grader) + 
m7*(neighborhood concentration of poverty;;) + ms*(neighborhood social capitalj,) + mo*(prior math score,;) + 


m10*(prior reading score;:) + ey 


where i represents students, 7 represents schools, ¢ represents time and ¢; represents the school-fixed effects. The 
model allows for a different trend for the four groups of schools included in the analyses. a1 represents the trend of 
the Success schools, 81 represents the trend for the AUSL schools, etc. The parameters a2 and a; represent the 
deviation with respect to the Success schools trend for the years 2014-5 and 2015-16. In other words whehter the 
outcome for the Success schools was higher or lower than what was expected given their prior trends. These two 
estimates are the values plotted in the figures in Chapter 3. Similarly, the parameter B2 and £3 represent the 
deviations with respect to the prior trend for the AUSL schools for the years 2014-15 and 2015-16, etc. 


Tables B.1 and B.2 show the raw means of the outcomes by the four groups of schools. 
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TABLE B.1 


Middle Grade Outcomes Means by Year and Group of Schools for the Analytic Sample 


Success Schools AUSL Schools 


Attendance 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
Reading GPA 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 

Math GPA 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 

Core GPA 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
Middle Grades On-Track Rates 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 
Study Habits 
2011-12 
2012-13 
2013-14 
2014-15 
2015-16 


Note: Survey measures are reported in standard deviation units. 
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Invited Schools 


Schools in Same 
Networks 


94.0% 
94.7% 
95.0% 
95.3% 


2.32 
2.36 
2.40 
2.49 
PSI] 


De) 
2.34 
2cSe) 
2.47 
2.52 


2.51 
22 
2.60 
2.69 
2.74 


57.3% 
50.9% 
52.9% 
58.0% 
60.3% 


0.23 
0.23 
0.23 
0.26 
0.25 


55 


54 


TABLE B.1: CONTINUED 
Middle Grade Outcomes Means by Year and Group of Schools for the Analytic Sample 


Success Schools AUSL Schools Invited Schools Schools in Same 
Networks 


Grit 
2011-12 


2012-13 0.03 
2013-14 0.07 
2014-15 0.08 
2015-16 0.09 
Peer Support for Academic Work 

2011-12 0.03 
2012-13 0.03 
2013-14 0.05 
2014-15 0.06 
2015-16 0.17 
Academic Personalism 

2011-12 0.19 
2012-13 0.11 
2013-14 0.13 
2014-15 0.19 
2015-16 0.20 
Student-Teacher Trust 

2011-12 0.19 
2012-13 0.14 
2013-14 0.11 
2014-15 ONS 
2015-16 0.14 
School Connectedness 

2011-12 0.11 
2012-13 0.15 
2013-14 0.15 
2014-15 0.18 
2015-16 0.16 


Appendix B 


TABLE B.2 
High School Choice Outcomes Means by Year and Group of Schools for the Analytic Sample 


Success Schools AUSL Schools Invited Schools Schools in Same 
Networks 


Average High School ACT Graduates Attended 


2011-12 


2012-13 17.8 
2013-14 We a7/ 
2014-15 17.9 


2015-16 


Average High School Graduation Rates Graduates Attended 


2011-12 


2012-13 72.8% 
2013-14 75.3% 
2014-15 74.5% 


2015-16 
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